Optical fingerprint sensor with folded light path

ABSTRACT

An optical fingerprint sensor module includes a light source configured to provide illumination light directed toward a finger to generate signal light scattered or reflected off of the finger, a photodiode array having a surface, an optically transparent spacer disposed over the surface of the photodiode array, a first mirror configured to reflect the signal light, a lens configured to receive and refract the signal light reflected by the first mirror, a member defining a pinhole disposed behind the lens and configured to transmit the signal light refracted by the lens, a second mirror disposed behind the pinhole and above the optically transparent spacer and configured to reflect the signal light transmitted through the pinhole toward the surface of the photodiode array, and electronic circuitries configured to process electrical signals generated by the photodiode array to produce an image of a fingerprint pattern of the finger.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation-in-part application of U.S. patentapplication Ser. No. 16/190,141, filed on Nov. 13, 2018, which is acontinuation-in-part application of U.S. patent application Ser. No.16/190,138, filed on Nov. 13, 2018, which claims the benefit of U.S.Provisional Patent Application No. 62/703,432, filed on Jul. 25, 2018,the contents of which are incorporated by reference in their entirety.

TECHNICAL FIELD

This disclosure relates to sensing of fingerprints and performing one ormore sensing operations of other parameter measurements in electronicdevices or systems, including portable computing devices such as mobiledevices, wearable devices, and larger systems.

BACKGROUND

Various sensors can be implemented in electronic devices or systems toprovide certain desired functions. A sensor that enables userauthentication is one example of sensors to protect personal data andprevent unauthorized access in various devices and systems includingportable or mobile computing devices (e.g., laptops, tablets,smartphones), gaming systems, various databases, information systems orlarger computer-controlled systems.

User authentication on an electronic device or system can be carried outthrough one or multiple forms of biometric identifiers, which can beused alone or in addition to conventional password authenticationmethods. A popular form of biometric identifiers is a person'sfingerprint pattern. A fingerprint sensor can be built into theelectronic device to read a user's fingerprint pattern so that thedevice can only be unlocked by an authorized user of the device throughauthentication of the authorized user's fingerprint pattern. Anotherexample of sensors for electronic devices or systems is a biomedicalsensor that detects a biological property of a user, e.g., a property ofa user's blood, the heartbeat, in wearable devices like wrist banddevices or watches. In general, different sensors can be provided inelectronic devices to achieve different sensing operations andfunctions.

Fingerprints can be used to authenticate users for accessing electronicdevices, computer-controlled systems, electronic databases orinformation systems, either used as a stand-alone authentication methodor in combination with one or more other authentication methods such asa password authentication method. For example, electronic devicesincluding portable or mobile computing devices, such as laptops,tablets, smartphones, and gaming systems can employ user authenticationmechanisms to protect personal data and prevent unauthorized access. Inanother example, a computer or a computer-controlled device or systemfor an organization or enterprise should be secured to allow onlyauthorized personnel to access in order to protect the information orthe use of the device or system for the organization or enterprise. Theinformation stored in portable devices and computer-controlleddatabases, devices or systems, may be personal in nature, such aspersonal contacts or phonebook, personal photos, personal healthinformation or other personal information, or confidential informationfor proprietary use by an organization or enterprise, such as businessfinancial information, employee data, trade secrets and otherproprietary information. If the security of the access to the electronicdevice or system is compromised, these data may be accessed by others,causing loss of privacy of individuals or loss of valuable confidentialinformation. Beyond security of information, securing access tocomputers and computer-controlled devices or systems also allowsafeguard the use of devices or systems that are controlled by computersor computer processors such as computer-controlled automobiles and othersystems such as ATMs.

Secured access to a device (e.g., a mobile device) or a system (e.g., anelectronic database and a computer-controlled system) can be achieved indifferent ways such as the use of user passwords. A password, however,may be easily to be spread or obtained and this nature of passwords canreduce the level of the security of passwords. Moreover, since a userneeds to remember a password in accessing password-protected electronicdevices or systems, in the event that the user forgets the password, theuser needs to undertake certain password recovery procedures to getauthenticated or otherwise to regain the access to the device or system.Such processes may be burdensome to users and have various practicallimitations and inconveniences. The personal fingerprint identificationcan be utilized to achieve the user authentication for enhancing thedata security while mitigating certain undesired effects associated withpasswords.

Electronic devices or systems, including portable or mobile computingdevices, may employ user authentication through one or multiple forms ofbiometric identifiers to protect personal or other confidential data andprevent unauthorized access. A biometric identifier can be used alone orin combination with a password authentication method to provide userauthentication. One form of biometric identifiers is a person'sfingerprint pattern. A fingerprint sensor can be built into anelectronic device or an information system to read a user's fingerprintpattern so that the device can only be unlocked by an authorized user ofthe device through authentication of the authorized user's fingerprintpattern.

SUMMARY

According to some embodiments, an optical fingerprint sensor moduleincludes a light source configured to provide illumination lightdirected toward a finger. A portion of the illumination light may bescattered or reflected off of the finger, thereby generating signallight. The optical fingerprint sensor module further includes aphotodiode array having a surface, and an optically transparent spacerdisposed over the surface of the photodiode array. The opticalfingerprint sensor module further includes a first mirror configured toreflect the signal light, and a lens configured to receive and refractthe signal light reflected by the first mirror. The lens has an opticalaxis that forms an angle with respect to a normal of the surface of thephotodiode array that is between 45 degrees and 135 degrees. The opticalfingerprint sensor module further includes a member defining a pinholedisposed behind the lens. The pinhole is configured to transmit thesignal light refracted by the lens. The optical fingerprint sensormodule further includes a second mirror disposed behind the pinhole andabove the optically transparent spacer. The second mirror is configuredto reflect the signal light transmitted through the pinhole toward thesurface of the photodiode array. The optical fingerprint sensor modulefurther includes electronic circuitries electrically coupled to thephotodiode array. The photodiode array is configured to convert thesignal light incident thereon into electrical signals. The electroniccircuitries are configured to process the electrical signals to producean image of a fingerprint pattern of the finger.

According to some embodiments, an optical fingerprint sensor module tobe disposed under an opaque border of a display screen for detecting afingerprint pattern of a finger placed adjacent a fingerprint sensingarea of the display screen includes a photodiode array having a surface,and an optically transparent spacer disposed over the surface of thephotodiode array. The optical fingerprint sensor module further includesa first mirror configured to reflect signal light scattered or reflectedoff of the finger and transmitted through the display screen, and a lensconfigured to receive and refract the signal light reflected by thefirst mirror. The lens has an optical axis that forms an angle withrespect to a normal of the surface of the photodiode array that isbetween 45 degrees and 135 degrees. The optical fingerprint sensormodule further includes a member defining a pinhole disposed behind thelens. The pinhole is configured to transmit the signal light refractedby the lens. The optical fingerprint sensor module further includes asecond mirror disposed behind the pinhole and above the opticallytransparent spacer. The second mirror is configured to reflect thesignal light transmitted through the pinhole toward the surface of thephotodiode array. The optical fingerprint sensor module further includeselectronic circuitries electrically coupled to the photodiode array. Thephotodiode array is configured to convert the signal light incidentthereon into electrical signals. The electronic circuitries areconfigured to process the electrical signals to produce an image of afingerprint pattern of the finger.

According to some embodiments, an electronic device includes a displayscreen. The display screen includes a fingerprint sensing area and anopaque border. The electronic device further includes a light sourceconfigured to provide illumination light directed toward a finger placedadjacent the fingerprint sensing area. A portion of the illuminationlight may be scattered or reflected off of the finger, therebygenerating signal light to be transmitted through the display screen.The electronic device further includes an optical fingerprint sensormodule positioned below the display screen under the opaque border. Theoptical fingerprint sensor module includes a photodiode array having asurface, and an optically transparent spacer disposed over the surfaceof the photodiode array. The optical fingerprint sensor module furtherincludes a first mirror configured to reflect the signal light, and alens configured to receive and refract the signal light reflected by thefirst mirror. The lens has an optical axis that forms an angle withrespect to a normal of the surface of the photodiode array that isbetween 45 degrees and 135 degrees. The optical fingerprint sensormodule further includes a member defining a pinhole disposed behind thelens. The pinhole is configured to transmit the signal light refractedby the lens. The optical fingerprint sensor module further includes asecond mirror disposed behind the pinhole and above the opticallytransparent spacer. The second mirror is configured to reflect thesignal light transmitted through the pinhole toward the surface of thephotodiode array. The optical fingerprint sensor module further includeselectronic circuitries electrically coupled to the photodiode array. Thephotodiode array is configured to convert the signal light incidentthereon into electrical signals. The electronic circuitries areconfigured to process the electrical signals to produce an image of afingerprint pattern of the finger.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example of a system with a fingerprintsensing module which can be implemented to include an opticalfingerprint sensor according to some embodiments.

FIGS. 2A and 2B illustrate an exemplary implementation of an electronicdevice having a touch sensing display screen assembly and an opticalfingerprint sensor module positioned underneath the touch sensingdisplay screen assembly according to some embodiments.

FIGS. 3A and 3B illustrate an example of a device that implements theoptical fingerprint sensor module illustrated in FIGS. 2A and 2Baccording to some embodiments.

FIGS. 4A and 4B show an exemplary implementation of an opticalfingerprint sensor module under the display screen assembly forimplementing the design illustrated in FIGS. 2A and 2B according to someembodiments.

FIGS. 5A-5C illustrate signal generation for the returned light from thesensing zone on the top sensing surface under two different opticalconditions to facilitate the understanding of the operation of anunder-screen optical fingerprint sensor module according to someembodiments.

FIGS. 6A-6C, 7, 8A-8B, 9, and 10A-10B illustrate example designs ofunder-screen optical fingerprint sensor modules according to someembodiments.

FIG. 11 illustrates imaging of the fingerprint sensing area on the toptransparent layer via an imaging module under different tilingconditions where an imaging device images the fingerprint sensing areaonto an optical sensor array and the imaging device may be opticallytransmissive or optically reflective according to some embodiments.

FIG. 12 is a flowchart illustrating an exemplary operation of afingerprint sensor for reducing or eliminating undesired contributionsfrom the background light in fingerprint sensing according to someembodiments.

FIG. 13 is a flowchart illustrating an exemplary process for operatingan under-screen optical fingerprint sensor module for capturing afingerprint pattern according to some embodiments.

FIGS. 14-16 illustrates exemplary operation processes for determiningwhether an object in contact with the LCD display screen is part of afinger of a live person by illuminating the finger with light in twodifferent light colors according to some embodiments.

FIGS. 17-23 illustrate examples and operations of an under-displayscreen optical fingerprint sensor module based on a pinhole-lensassembly according to some embodiments.

FIGS. 24A-24B, 25A-25C, and 26 illustrate examples of an invisibleunder-display screen optical fingerprint sensor module using an opticalcoupler according to some embodiments.

FIG. 27 illustrates an exemplary implementation of integrating anoptical fingerprint sensor module with a display screen according tosome embodiments.

FIG. 28 shows an exemplary optical fingerprint sensor module that may beintegrated with a display screen, as illustrated in FIG. 27, accordingsome embodiments.

FIG. 29 shows an exemplary structure of a display screen integrated withan optical fingerprint sensor module, as illustrated in FIGS. 27 and 28,according to some embodiments.

FIG. 30 illustrates an optical fingerprint sensor module according tosome embodiments.

FIG. 31 illustrates an optical fingerprint sensor module according tosome embodiments.

FIG. 32 illustrates an optical fingerprint sensor module according tosome embodiments.

FIG. 33 illustrates different light signals that may be present in adevice that implements the under-display screen optical fingerprintsensing design as illustrated in FIGS. 27-32.

FIGS. 34A and 34B illustrate an optical fingerprint sensor moduleaccording to some embodiments.

FIGS. 35A and 35B illustrate an optical fingerprint sensor moduleaccording to some embodiments.

FIGS. 36A and 36B illustrate an optical fingerprint sensor moduleaccording to some embodiments.

DETAILED DESCRIPTION

Electronic devices or systems may be equipped with fingerprintauthentication mechanisms to improve the security for accessing thedevices. Such electronic devices or system may include, portable ormobile computing devices, e.g., smartphones, tablet computers,wrist-worn devices and other wearable or portable devices, largerelectronic devices or systems, e.g., personal computers in portableforms or desktop forms, ATMs, various terminals to various electronicsystems, databases, or information systems for commercial orgovernmental uses, motorized transportation systems includingautomobiles, boats, trains, aircraft and others.

Fingerprint sensing is useful in mobile applications and otherapplications that use or require secure access. For example, fingerprintsensing can be used to provide secure access to a mobile device andsecure financial transactions including online purchases. It isdesirable to include robust and reliable fingerprint sensing suitablefor mobile devices and other applications. In mobile, portable orwearable devices, it is desirable for fingerprint sensors to minimize oreliminate the footprint for fingerprint sensing given the limited spaceon those devices, especially considering the demands for a maximumdisplay area on a given device. Many implementations of capacitivefingerprint sensors must be implemented on the top surface of a devicedue to the near-field interaction requirement of capacitive sensing.

Optical sensing modules can be designed to mitigate the above and otherlimitations in the capacitive fingerprint sensors and to achieveadditional technical advantages. For example, in implementing an opticalfingerprint sensing device, the light carrying fingerprint imagininginformation can be directed over distance to an optical detector arrayof optical detectors for detecting the fingerprint without being limitedto the near-field sensing in a capacitive sensor. In particular, lightcarrying fingerprint imagining information can be directed to transmitthrough the top cover glass commonly used in many display screens suchas touch sensing screens and other structures and may be directedthrough folded or complex optical paths to reach the optical detectorarray, thus allowing for flexibility in placing an optical fingerprintsensor in a device that is not available for a capacitive fingerprintsensor. Optical fingerprint sensor modules based on the technologiesdisclosure herein can be an under-screen optical fingerprint sensormodule that is placed below a display screen to capture and detect lightfrom a finger placed on or above the top sensing surface of the screen.As disclosed herein, optical sensing can also be used to, in addition todetecting and sensing a fingerprint pattern, optically detect otherparameters associated with a user or a user action, such as whether adetected fingerprint is from a finger of a live person and to provideanti-spoofing mechanism, or certain biological parameters of the user.

I. Overview of Under-Display Optical Sensing Modules

The optical sensing technology and examples of implementations describedin this disclosure provide an optical fingerprint sensor module thatuses, at least in part, the light from a display screen as theillumination probe light to illuminate a fingerprint sensing area on thetouch sensing surface of the display screen to perform one or moresensing operations based on optical sensing of such light. A suitabledisplay screen for implementing the disclosed optical sensor technologycan be based on various display technologies or configurations,including, a liquid crystal display (LCD) screen using a backlight toprovide white light illumination to the LCD pixels and matched opticalfilters to effectuate colored LCD pixels, or a display screen havinglight emitting display pixels without using backlight where eachindividual pixel generates light for forming a display image on thescreen such as an organic light emitting diode (OLED) display screens,or electroluminescent display screens. The specific examples providedbelow are directed to integration of under-screen optical sensingmodules with LCD screens and thus contain certain technical detailsassociated with LCD screens although various aspects of the disclosedtechnology are applicable to OLED screens and other display screens.

A portion of the light produced by a display screen for displayingimages necessarily passes through the top surface of the display screenin order to be viewed by a user. A finger in touch with or near the topsurface interacts with the light at the top surface to cause thereflected or scattered light at the surface area of the touch to carryspatial image information of the finger. Such reflected or scatteredlight carrying the spatial image information of the finger returns tothe display panel underneath the top surface. In touch sensing displaydevices, for example, the top surface is the touch sensing interfacewith the user and this interaction between the light for displayingimages and the user finger or hand constantly occurs but suchinformation-carrying light returning back to the display panel islargely wasted and is not used in various touch sensing devices. Invarious mobile or portable devices with touch sensing displays andfingerprint sensing functions, a fingerprint sensor tends to be aseparate device from the display screen, either placed on the samesurface of the display screen at a location outside the display screenarea such as in some models of Apple iPhones and Samsung smartphones, orplaced on the backside of a smartphone, such as some models of smartphones by Huawei, Lenovo, Xiaomi or Google, to avoid taking up valuablespace for placing a large display screen on the front side. Thosefingerprint sensors are separate devices from the display screens andthus need to be compact to save space for the display screens and otherfunctions while still providing reliable and fast fingerprint sensingwith a spatial image resolution above a certain acceptable level.However, the need to be compact and small for designing a fingerprintsensor and the need to provide a high spatial image resolution incapturing a fingerprint pattern are in direct conflict with each otherin many fingerprint sensors because a high spatial image resolution incapturing a fingerprint pattern in based on various suitable fingerprintsensing technologies (e.g., capacitive touch sensing or optical imaging)requires a large sensor area with a large number of sensing pixels.

The sensor technology and examples of implementations of the sensortechnology described in this disclosure provide an optical fingerprintsensor module that uses, at least in part, the light from a displayscreen as the illumination probe light to illuminate a fingerprintsensing area on the touch sensing surface of the display screen toperform one or more sensing operations based on optical sensing of suchlight in some implementations, or designated illumination or probe lightfor optical sensing from one or more designated illumination lightsources separate from the display light for optical sensing in otherimplementations, or background light for optical sensing in certainimplementations.

In the disclosed examples for integrating an optical sensing module to aLCD screen based on the disclosed optical sensor technology, the underLCD optical sensor can be used to detect a portion of the light that isused for displaying images in a LCD screen where such a portion of thelight for the display screen may be the scattered light, reflected lightor some stray light. For example, in some implementations, the imagelight of the LCD screen based on backlighting may be reflected orscattered back into the LCD display screen as returned light whenencountering an object such as a user finger or palm, or a user pointerdevice like a stylus. Such returned light can be captured for performingone or more optical sensing operations using the disclosed opticalsensor technology. Due to the use of the light from LCD screen foroptical sensing, an optical fingerprint sensor module based on thedisclosed optical sensor technology is specially designed to beintegrated to the LCD display screen in a way that maintains the displayoperations and functions of the LCD display screen without interferencewhile providing optical sensing operations and functions to enhanceoverall functionality, device integration and user experience of anelectronic device or system such as a smart phone, a tablet, or amobile/wearable device.

In addition, in various implementations of the disclosed optical sensingtechnology, one or more designated probe light sources may be providedto produce additional illumination probe light for the optical sensingoperations by the under-LCD screen optical sensing module. In suchapplications, the light from the backlighting of the LCD screen and theprobe light from the one or more designated probe light sourcescollectively form the illumination light for optical sensing operations.

Regarding the additional optical sensing functions beyond fingerprintdetection, the optical sensing may be used to measure other parameters.For example, the disclosed optical sensor technology can measure apattern of a palm of a person given the large touch area available overthe entire LCD display screen (in contrast, some designated fingerprintsensors such as the fingerprint senor in the home button of Apple'siPhone/iPad devices have a rather small and designated off-screenfingerprint sensing area that is highly limited in the sensing area sizethat may not be suitable for sensing large patterns). For yet anotherexample, the disclosed optical sensor technology can be used not only touse optical sensing to capture and detect a pattern of a finger or palmthat is associated with a person, but also to use optical sensing orother sensing mechanisms to detect whether the captured or detectedpattern of a fingerprint or palm is from a live person's hand by a “livefinger” detection mechanism, which may be based on, for example, thedifferent optical absorption behaviors of the blood at different opticalwavelengths, the fact that a live person's finger tends to be moving orstretching due to the person's natural movement or motion (eitherintended or unintended) or pulsing when the blood flows through theperson's body in connection with the heartbeat. In one implementation,the optical fingerprint sensor module can detect a change in thereturned light from a finger or palm due to the heartbeat/blood flowchange and thus to detect whether there is a live heartbeat in theobject presented as a finger or palm. The user authentication can bebased on the combination of the both the optical sensing of thefingerprint/palm pattern and the positive determination of the presenceof a live person to enhance the access control. For yet another example,the optical fingerprint sensor module may include a sensing function formeasuring a glucose level or a degree of oxygen saturation based onoptical sensing in the returned light from a finger or palm. As yetanother example, as a person touches the LCD display screen, a change inthe touching force can be reflected in one or more ways, includingfingerprint pattern deforming, a change in the contacting area betweenthe finger and the screen surface, fingerprint ridge widening, or achange in the blood flow dynamics. Those and other changes can bemeasured by optical sensing based on the disclosed optical sensortechnology and can be used to calculate the touch force. This touchforce sensing can be used to add more functions to the opticalfingerprint sensor module beyond the fingerprint sensing.

With respect to useful operations or control features in connection withthe touch sensing aspect of the LCD display screen, the disclosedoptical sensor technology can provide triggering functions or additionalfunctions based on one or more sensing results from the opticalfingerprint sensor module to perform certain operations in connectionwith the touch sensing control over the LCD display screen. For example,the optical property of a finger skin (e.g., the index of refraction)tends to be different from other artificial objects. Based on this, theoptical fingerprint sensor module may be designed to selectively receiveand detect returned light that is caused by a finger in touch with thesurface of the LCD display screen while returned light caused by otherobjects would not be detected by the optical fingerprint sensor module.This object-selective optical detection can be used to provide usefuluser controls by touch sensing, such as waking up the smartphone ordevice only by a touch via a person's finger or palm while touches byother objects would not cause the device to wake up for energy efficientoperations and to prolong the battery use. This operation can beimplemented by a control based on the output of the optical fingerprintsensor module to control the waking up circuitry operation of the LCDdisplay screen which, the LCD pixels are put in a “sleep” mode by beingturned off (and the LCD backlighting is also turned off)while one ormore illumination light sources (e.g., LEDs) for the under-LCD paneloptical fingerprint sensor module are turned on in a flash mode tointermittently emit flash light to the screen surface for sensing anytouch by a person's finger or palm. Under this design, the opticalfingerprint sensor module operates the one or more illumination lightsources to produce the “sleep” mode wake-up sensing light flashes sothat the optical fingerprint sensor module can detect returned light ofsuch wake-up sensing light caused by the finger touch on the LCD displayscreen and, upon a positive detection, the LCD backlighting and the LCDdisplay screen are turned on or “woken up”. In some implementations, thewake-up sensing light can be in the infrared invisible spectral range soa user will not experience any visual of a flash light. The LCD displayscreen operation can be controlled to provide an improved fingerprintsensing by eliminating background light for optical sensing of thefingerprint. In one implementation, for example, each display scan framegenerates a frame of fingerprint signals. If, two frames of fingerprintsignals with the display are generated in one frame when the LCD displayscreen is turned on and in the other frame when the LCD display screenis turned off, the subtraction between those two frames of signals canbe used to reduce the ambient background light influence. By operatingthe fingerprint sensing frame rate is at one half of the display framerate in some implementations, the background light noise in fingerprintsensing can be reduced.

An optical fingerprint sensor module based on the disclosed opticalsensor technology can be coupled to the backside of the LCD displayscreen without requiring creation of a designated area on the surfaceside of the LCD display screen that would occupy a valuable devicesurface real estate in some electronic devices such as a smartphone, atablet or a wearable device. This aspect of the disclosed technology canbe used to provide certain advantages or benefits in both device designsand product integration or manufacturing.

In some implementations, an optical fingerprint sensor module based onthe disclosed optical sensor technology can be configured as anon-invasive module that can be easily integrated to a display screenwithout requiring changing the design of the LCD display screen forproviding a desired optical sensing function such as fingerprintsensing. In this regard, an optical fingerprint sensor module based onthe disclosed optical sensor technology can be independent from thedesign of a particular LCD display screen design due to the nature ofthe optical fingerprint sensor module: the optical sensing of such anoptical fingerprint sensor module is by detecting the light that isemitted by the one or more illumination light sources of the opticalfingerprint sensor module and is returned from the top surface of thedisplay area, and the disclosed optical fingerprint sensor module iscoupled to the backside of the LCD display screen as a under-screenoptical fingerprint sensor module for receiving the returned light fromthe top surface of the display area and thus does not require a specialsensing port or sensing area that is separate from the display screenarea. Accordingly, such an under-screen optical fingerprint sensormodule can be used to combine with a LCD display screen to provideoptical fingerprint sensing and other sensor functions on an LCD displayscreen without using a specially designed LCD display screen withhardware especially designed for providing such optical sensing. Thisaspect of the disclosed optical sensor technology enables a wide rangeof LCD display screens in smartphones, tablets or other electronicdevices with enhanced functions from the optical sensing of thedisclosed optical sensor technology.

For example, for an existing phone assembly design that does not providea separate fingerprint sensor as in certain Apple iPhones or SamsungGalaxy smartphones, such an existing phone assembly design can integratethe under-screen optical fingerprint sensor module as disclosed hereinwithout changing the touch sensing-display screen assembly to provide anadded on-screen fingerprint sensing function. Because the disclosedoptical sensing does not require a separate designated sensing area orport as in the case of certain Apple iPhones/Samsung Galaxy phones witha front fingerprint senor outside the display screen area, or somesmartphones with a designated rear fingerprint sensor on the backsidelike in some models by Huawei, Xiaomi, Google or Lenovo, the integrationof the on-screen fingerprint sensing disclosed herein does not require asubstantial change to the existing phone assembly design or the touchsensing display module that has both the touch sensing layers and thedisplay layers. Based on the disclosed optical sensing technology inthis document, no external sensing port and no extern hardware buttonare needed on the exterior of a device are needed for adding thedisclosed optical fingerprint sensor module for fingerprint sensing. Theadded optical fingerprint sensor module and the related circuitry areunder the display screen inside the phone housing and the fingerprintsensing can be conveniently performed on the same touch sensing surfacefor the touch screen.

For another example, due to the above described nature of the opticalfingerprint sensor module for fingerprint sensing, a smartphone thatintegrates such an optical fingerprint sensor module can be updated withimproved designs, functions and integration mechanism without affectingor burdening the design or manufacturing of the LCD display screens toprovide desired flexibility to device manufacturing andimprovements/upgrades in product cycles while maintaining theavailability of newer versions of optical sensing functions tosmartphones, tablets or other electronic devices using LCD displayscreens. Specifically, the touch sensing layers or the LCD displaylayers may be updated in the next product release without adding anysignificant hardware change for the fingerprint sensing feature usingthe disclosed under-screen optical fingerprint sensor module. Also,improved on-screen optical sensing for fingerprint sensing or otheroptical sensing functions by such an optical fingerprint sensor modulecan be added to a new product release by using a new version of theunder-screen optical fingerprint sensor module without requiringsignificant changes to the phone assembly designs, including addingadditional optical sensing functions.

The above and other features of the disclosed optical sensor technologycan be implemented to provide a new generation of electronic deviceswith improved fingerprint sensing and other sensing functions,especially for smartphones, tablets and other electronic devices withLCD display screens to provide various touch sensing operations andfunctions and to enhance the user experience in such devices. Thefeatures for optical fingerprint sensor modules disclosed herein may beapplicable to various display panels based on different technologiesincluding both LCD and OLED displays. The specific examples below aredirected to LCD display panels and optical fingerprint sensor modulesplaced under LCD display panels.

In implementations of the disclosed technical features, additionalsensing functions or sensing modules, such as a biomedical sensor, e.g.,a heartbeat sensor in wearable devices like wrist band devices orwatches, may be provided. In general, different sensors can be providedin electronic devices or systems to achieve different sensing operationsand functions.

The disclosed technology can be implemented to provide devices, systems,and techniques that perform optical sensing of human fingerprints andauthentication for authenticating an access attempt to a lockedcomputer-controlled device such as a mobile device or acomputer-controlled system, that is equipped with a fingerprintdetection module. The disclosed technology can be used for securingaccess to various electronic devices and systems, including portable ormobile computing devices such as laptops, tablets, smartphones, andgaming devices, and other electronic devices or systems such aselectronic databases, automobiles, bank ATMs, etc.

II. Design Examples of Under-Display Optical Sensing Modules

In the following sections, examples of various designs for anunder-screen optical fingerprint sensor module for collecting an opticalsignal to the optical detectors and providing desired optical imagingsuch as a sufficient imaging resolution. Specific examples of opticalimaging designs for the under-screen optical sensing modules areprovided below, including optical imaging designs without an imaginglens, optical imaging designs with at least one imaging lens and opticalimaging designs based on combining a pinhole and an imaging lens in apinhole-lens assembly for improved optical imaging and compact opticalfingerprint sensor module packaging.

FIG. 1 is a block diagram of an example of a system 180 with afingerprint sensing module 180 including a fingerprint sensor 181 whichcan be implemented to include an optical fingerprint sensor based on theoptical sensing of fingerprints as disclosed in this document. Thesystem 180 includes a fingerprint sensor control circuit 184, and adigital processor 186 which may include one or more processors forprocessing fingerprint patterns and determining whether an inputfingerprint pattern is one for an authorized user. The fingerprintsensing system 180 uses the fingerprint sensor 181 to obtain afingerprint and compares the obtained fingerprint to a storedfingerprint to enable or disable functionality in a device or system 188that is secured by the fingerprint sensing system 180. In operation, theaccess to the device 188 is controlled by the fingerprint processingprocessor 186 based on whether the captured user fingerprint is from anauthorized user. As illustrated, the fingerprint sensor 181 may includemultiple fingerprint sensing pixels such as pixels 182A-182E thatcollectively represent at least a portion of a fingerprint. For example,the fingerprint sensing system 180 may be implemented at an ATM as thesystem 188 to determine the fingerprint of a customer requesting toaccess funds or other transactions. Based on a comparison of thecustomer's fingerprint obtained from the fingerprint sensor 181 to oneor more stored fingerprints, the fingerprint sensing system 180 may,upon a positive identification, cause the ATM system 188 to grant therequested access to the user account, or, upon a negativeidentification, may deny the access. For another example, the device orsystem 188 may be a smartphone or a portable device and the fingerprintsensing system 180 is a module integrated to the device 188. For anotherexample, the device or system 188 may be a gate or secured entrance to afacility or home that uses the fingerprint sensor 181 to grant or denyentrance. For yet another example, the device or system 188 may be anautomobile or other vehicle that uses the fingerprint sensor 181 to linkto the start of the engine and to identify whether a person isauthorized to operate the automobile or vehicle.

As a specific example, FIGS. 2A and 2B illustrate one exemplaryimplementation of an electronic device 200 having a touch sensingdisplay screen assembly and an optical fingerprint sensor modulepositioned underneath the touch sensing display screen assembly. In thisparticular example, the display technology can be implemented by a LCDdisplay screen with backlight for optically illuminating the LCD pixelsor another display screen having light emitting display pixels withoutusing backlight (e.g., an OLED display screen). The electronic device200 can be a portable device such as a smartphone or a tablet and can bethe device 188 as shown in FIG. 1.

FIG. 2A shows the front side of the device 200 which may resemble somefeatures in some existing smartphones or tablets. The device screen ison the front side of the device 200 occupying either entirety, amajority or a significant portion of the front side space and thefingerprint sensing function is provided on the device screen, e.g., oneor more sensing areas for receiving a finger on the device screen. As anexample, FIG. 2A shows a fingerprint sensing zone in the device screenfor a finger to touch which may be illuminated as a visibly identifiablezone or area for a user to place a finger for fingerprint sensing. Sucha fingerprint sensing zone can function like the rest of the devicescreen for displaying images. As illustrated, the device housing of thedevice 200 may have, in various implementations, side facets thatsupport side control buttons that are common in various smartphones onthe market today. Also, one or more optional sensors may be provided onthe front side of the device 200 outside the device screen asillustrated by one example on the left upper corner of the devicehousing in FIG. 2A.

FIG. 2B shows an example of the structural construction of the modulesin the device 200 relevant to the optical fingerprint sensing disclosedin this document. The device screen assembly shown in FIG. 2B includes,e.g., the touch sensing screen module with touch sensing layers on thetop, and a display screen module with display layers located underneaththe touch sensing screen module. An optical fingerprint sensor module iscoupled to, and located underneath, the display screen assembly moduleto receive and capture the returned light from the top surface of thetouch sensing screen module and to guide and image the returned lightonto an optical sensor array of optical sensing pixels or photodetectorswhich convert the optical image in the returned light into pixel signalsfor further processing. Underneath the optical fingerprint sensor moduleis the device electronics structure containing certain electroniccircuits for the optical fingerprint sensor module and other parts inthe device 200. The device electronics may be arranged inside the devicehousing and may include a part that is under the optical fingerprintsensor module as shown in FIG. 2B.

In implementations, the top surface of the device screen assembly can bea surface of an optically transparent layer serving as a user touchsensing surface to provide multiple functions, such as (1) a displayoutput surface through which the light carrying the display imagespasses through to reach a viewer's eyes, (2) a touch sensing interfaceto receive a user's touches for the touch sensing operations by thetouch sensing screen module, and (3) an optical interface for on-screenfingerprint sensing (and possibly one or more other optical sensingfunctions). This optically transparent layer can be a rigid layer suchas a glass or crystal layer or a flexible layer.

One example of a display screen is an LCD display having LCD layers anda thin film transistor (TFT) structure or substrate. A LCD display panelis a multi-layer liquid crystal display (LCD) module that includes LCDdisplay backlighting light sources (e.g., LED lights) emitting LCDillumination light for LCD pixels, a light waveguide layer to guide thebacklighting light, and LCD structure layers which can include, e.g., alayer of liquid crystal

(LC) cells, LCD electrodes, transparent conductive ITO layer, an opticalpolarizer layer, a color filter layer, and a touch sensing layer. TheLCD module also includes a backlighting diffuser underneath the LCDstructure layers and above the light waveguide layer to spatially spreadthe backlighting light for illuminating the LCD display pixels, and anoptical reflector film layer underneath the light waveguide layer torecycle backlighting light towards the LCD structure layers for improvedlight use efficiency and the display brightness. For optical sensing,one or more separate illumination light sources are provided and areoperated independently from the backlighting light sources of the LCDdisplay module.

Referring to FIG. 2B, the optical fingerprint sensor module in thisexample is placed under the LCD display panel to capture the returnedlight from the top touch sensing surface and to acquire high resolutionimages of fingerprint patterns when user's finger is in touch with asensing area on the top surface. In other implementations, the disclosedunder-screen optical fingerprint sensor module for fingerprint sensingmay be implemented on a device without the touch sensing feature.

FIGS. 3A and 3B illustrate an example of a device that implements theoptical fingerprint sensor module in FIGS. 2A and 2B. FIG. 3A shows across sectional view of a portion of the device containing theunder-screen optical fingerprint sensor module. FIG. 3B shows, on theleft, a view of the front side of the device with the touch sensingdisplay indicating a fingerprint sensing area on the lower part of thedisplay screen, and on the right, a perspective view of a part of thedevice containing the optical fingerprint sensor module that is underthe device display screen assembly. FIG. 3B also shows an example of thelayout of the flexible tape with circuit elements.

In the design examples in FIGS. 2A-2B, and 3A-3B, the opticalfingerprint sensor design is different from some other fingerprintsensor designs using a separate fingerprint sensor structure from thedisplay screen with a physical demarcation between the display screenand the fingerprint sensor (e.g., a button like structure in an openingof the top glass cover in some mobile phone designs) on the surface ofthe mobile device. In the illustrated designs here, the opticalfingerprint sensor for detecting fingerprint sensing and other opticalsignals are located under the top cover glass or layer (e.g., FIG. 3A)so that the top surface of the cover glass serves as the top surface ofthe mobile device as a contiguous and uniform glass surface across boththe display screen layers and the optical detector sensor that arevertically stacked and vertically overlap. This design example forintegrating optical fingerprint sensing and the touch sensitive displayscreen under a common and uniform surface provides benefits, includingimproved device integration, enhanced device packaging, enhanced deviceresistance to exterior elements, failure and wear and tear, and enhanceduser experience over the ownership period of the device.

Referring back to FIGS. 2A and 2B, the illustrated under-screen opticalfingerprint sensor module for on-screen fingerprint sensing may beimplemented in various configurations.

In one implementation, a device based on the above design can bestructured to include a device screen a that provides touch sensingoperations and includes a LCD display panel structure for forming adisplay image, a top transparent layer formed over the device screen asan interface for being touched by a user for the touch sensingoperations and for transmitting the light from the display structure todisplay images to a user, and an optical fingerprint sensor modulelocated below the display panel structure to receive light that returnsfrom the top transparent layer to detect a fingerprint.

This device and other devices disclosed herein can be further configuredto include various features.

For example, a device electronic control module can be included in thedevice to grant a user's access to the device if a detected fingerprintmatches a fingerprint an authorized user. In addition, the opticalfingerprint sensor module is configured to, in addition to detectingfingerprints, also detect a biometric parameter different form afingerprint by optical sensing to indicate whether a touch at the toptransparent layer associated with a detected fingerprint is from a liveperson, and the device electronic control module is configured to granta user's access to the device if both (1) a detected fingerprint matchesa fingerprint an authorized user and (2) the detected biometricparameter indicates the detected fingerprint is from a live person. Thebiometric parameter can include, e.g., whether the finger contains ablood flow, or a heartbeat of a person.

For example, the device can include a device electronic control modulecoupled to the display panel structure to supply power to the lightemitting display pixels and to control image display by the displaypanel structure, and, in a fingerprint sensing operation, the deviceelectronic control module operates to turn off the light emittingdisplay pixels in one frame to and turn on the light emitting displaypixels in a next frame to allow the optical sensor array to capture twofingerprint images with and without the illumination by the lightemitting display pixels to reduce background light in fingerprintsensing.

For another example, a device electronic control module may be coupledto the display panel structure to supply power to the LCD display paneland to turn off power to the backlighting of the LCD display panel in asleep mode, and the device electronic control module may be configuredto wake up the display panel structure from the sleep mode when theoptical fingerprint sensor module detects the presence of a person'sskin at the designated fingerprint sensing region of the top transparentlayer. More specifically, in some implementations, the device electroniccontrol module can be configured to operate one or more illuminationlight sources in the optical fingerprint sensor module to intermittentlyemit light, while turning off power to the LCD display panel (in thesleep mode), to direct the intermittently emitted illumination light tothe designated fingerprint sensing region of the top transparent layerfor monitoring whether there is a person's skin in contact with thedesignated fingerprint sensing region for waking up the device from thesleep mode.

For another example, the device can include a device electronic controlmodule coupled to the optical fingerprint sensor module to receiveinformation on multiple detected fingerprints obtained from sensing atouch of a finger and the device electronic control module is operatedto measure a change in the multiple detected fingerprints and determinesa touch force that causes the measured change. For instance, the changemay include a change in the fingerprint image due to the touch force, achange in the touch area due to the touch force, or a change in spacingof fingerprint ridges.

For another example, the top transparent layer can include a designatedfingerprint sensing region for a user to touch with a finger forfingerprint sensing and the optical fingerprint sensor module below thedisplay panel structure can include a transparent block in contact withthe display panel substrate to receive light that is emitted from thedisplay panel structure and returned from the top transparent layer, anoptical sensor array that receives the light and an optical imagingmodule that images the received light in the transparent block onto theoptical sensor array. The optical fingerprint sensor module can bepositioned relative to the designated fingerprint sensing region andstructured to selectively receive returned light via total internalreflection at the top surface of the top transparent layer when incontact with a person's skin while not receiving the returned light fromthe designated fingerprint sensing region in absence of a contact by aperson's skin.

For yet another example, the optical fingerprint sensor module can bestructured to include an optical wedge located below the display panelstructure to modify a total reflection condition on a bottom surface ofthe display panel structure that interfaces with the optical wedge topermit extraction of light out of the display panel structure throughthe bottom surface, an optical sensor array that receives the light fromthe optical wedge extracted from the display panel structure, and anoptical imaging module located between the optical wedge and the opticalsensor array to image the light from the optical wedge onto the opticalsensor array.

Specific examples of under-screen optical fingerprint sensor modules foron-screen fingerprint sensing are provided below.

FIGS. 4A and 4B show an example of one implementation of an opticalfingerprint sensor module under the display screen assembly forimplementing the design in FIGS. 2A and 2B. The device illustrated inFIGS. 4A-4B includes a display assembly 423 with a top transparent layer431 formed over the device screen assembly 423 as an interface for beingtouched by a user for the touch sensing operations and for transmittingthe light from the display structure to display images to a user. Thistop transparent layer 431 can be a cover glass or a crystal material insome implementations. The device screen assembly 423 can include a LCDdisplay module 433 under the top transparent layer 431. The LCD displaylayers allow partial optical transmission so light from the top surfacecan partially transmit through the LCD display layers to reach theunder-LCD optical fingerprint sensor module. For example, LCD displaylayers include electrodes and wiring structure optically acting as anarray of holes and light scattering objects. A device circuit module 435may be provided under the LCD display panel to control operations of thedevice and perform functions for the user to operate the device.

The optical fingerprint sensor module 702 in this particularimplementation example is placed under LCD display module 433. One ormore illumination light sources, e.g., an illumination light source 436under the LCD display module 433 or/and another one or more illuminationlight sources located under the top cover glass 431, are provided forproviding the illumination light or probe light for the optical sensingby the optical fingerprint sensor module 702 and can be controlled toemit light to at least partially pass through the LCD display module 433to illuminate the fingerprint sensing zone 615 on the top transparentlayer 431 within the device screen area for a user to place a fingertherein for fingerprint identification. The illumination light from theone or more illumination light sources 436 can be directed to thefingerprint sensing area 615 on the top surface as if such illuminationlight is from a fingerprint illumination light zone 613. Another one ormore illumination light sources may be located under the top cover glass431 and may be placed adjacent to the fingerprint sensing area 615 onthe top surface to direct produced illumination light to reach the topcover glass 433 without passing through the LCD display module 433. Insome designs, one or more illumination light sources may be locatedabove the bottom surface of the top cover glass 431 to direct producedillumination light to reach the fingerprint sensing region above the topsurface of the top cover glass 433 without necessarily passing throughthe top cover glass 431, e.g., directing illuminating the finger abovethe top cover glass 431.

As illustrated in FIG. 4A, a finger 445 is placed in the illuminatedfingerprint sensing zone 615 as the effective sensing zone forfingerprint sensing. A portion of the reflected or scattered light inthe zone 615 is directed into the optical fingerprint sensor moduleunderneath the LCD display module 433 and a photodetector sensing arrayinside the optical fingerprint sensor module receives such light andcaptures the fingerprint pattern information carried by the receivedlight. The one or more illumination light sources 436 are separate fromthe backlighting sources for the LCD display module and are operatedindependently from the backlighting light sources of the LCD displaymodule.

In this design of using one or more illumination light sources 436 toprovide the illumination light for optical fingerprint sensing, eachillumination light source 436 maybe controlled in some implementationsto turn on intermittently with a relatively low cycle to reduce thepower used for the optical sensing operations. The fingerprint sensingoperation can be implemented in a two-step process in someimplementations: first, the one or more illumination light sources 436are turned on in a flashing mode without turning on the LCD displaypanel to use the flashing light to sense whether a finger touches thesensing zone 615 and, once a touch in the zone 615 is detected, theoptical sensing module is operated to perform the fingerprint sensingbased on optical sensing and the LCD display panel may be turned on.

In the example in FIG. 4B, the under-screen optical fingerprint sensormodule includes a transparent block 701 that is coupled to the displaypanel to receive the returned light from the top surface of the deviceassembly, and an optical imaging block 702 that performs the opticalimaging and imaging capturing. Light from the one or more illuminationlight sources 436, after reaching the cover top surface, e.g., the covertop surface at the sensing area 615 where a user finger touches or islocated without touching the cover top surface, is reflected orscattered back from the cover top surface in a design in which theillumination light source 436 is located to direct the illuminationlight to first transmit through the top cover glass 431 to reach thefinger. When fingerprint ridges in contact of the cover top surface inthe sensing area 615, the light reflection under the fingerprint ridgesis different, due to the presence of the skin or tissue of the finger incontact at that location, from the light reflection at another locationunder the fingerprint valley, where the skin or tissue of the finger isabsent. This difference in light reflection conditions at the locationsof the ridges and valleys in the touched finger area on the cover topsurface forms an image representing an image or spatial distribution ofthe ridges and valleys of the touched section of the finger. Thereflection light is directed back towards the LCD display module 433,and, after passing through the small holes of the LCD display module433, reaches the interface with the low index optically transparentblock 701 of the optical fingerprint sensor module. The low indexoptically transparent block 701 is constructed to have a refractiveindex less than a refractive index of the LCD display panel so that thereturned light can be extracted out of the LCD display panel into theoptically transparent block 701. Once the returned light is receivedinside the optically transparent block 701, such received light entersthe optical imaging unit as part of the imaging sensing block 702 and isimaged onto the photodetector sensing array or optical sensing arrayinside the block 702. The light reflection differences betweenfingerprint ridges and valleys create the contrast of the fingerprintimage. As shown in FIG. 4B, a control circuit 704 (e.g., amicrocontroller or MCU) is coupled to the imaging sensing block 702 andto other circuitry such as the device main processor 705 on a maincircuit board.

In this particular example, the optical light path design is structuredso that the illumination light enters the cover top surface within thetotal reflection angles on the top surface between the substrate and airinterface and, therefore, the reflected light is collected mosteffectively by the imaging optics and imaging sensor array in the block702. In this design, the image of the fingerprint ridge/valley areaexhibits a maximum contrast due to the total internal reflectioncondition at each finger valley location where the finger tissue doesnot touch the top cover surface of the top cover glass 431. Someimplementations of such an imaging system may have undesired opticaldistortions that would adversely affect the fingerprint sensing.Accordingly, the acquired image may be further corrected by a distortioncorrection during the imaging reconstruction in processing the outputsignals of the optical sensor array in the block 702 based on theoptical distortion profile along the light paths of the returned lightat the optical sensor array. The distortion correction coefficients canbe generated by images captured at each photodetector pixel by scanninga test image pattern one line pixel at a time, through the whole sensingarea in both X direction lines and Y direction lines. This correctionprocess can also use images from tuning each individual pixel on one ata time, and scanning through the whole image area of the photodetectorarray. This correction coefficients only need to be generated one timeafter assembly of the sensor.

The background light from environment (e.g., sunlight or roomillumination light) may enter the image sensor through the LCD panel topsurface, and through holes in the LCD display assembly 433. Suchbackground light can create a background baseline in the interestedimages from a finger and thus may undesirably degrade the contrast of acaptured image. Different methods can be used to reduce this undesiredbaseline intensity caused by the background light. One example is totune on and off the illumination light source 436 at a certainillumination modulation frequency f and the image sensor accordinglyacquires the received images at the same illumination modulationfrequency by phase synchronizing the light source driving pulse andimage sensor frame. Under this operation, only one of the image phasescontain light from the light source. In implementing this technique, theimaging capturing can be timed to capture images with the illuminationlight on at even (or odd) frames while turning off the illuminationlight at odd (or even) frames and, accordingly, subtracting even and oddframes can be used to obtain an image which is mostly formed by lightemitted from the modulated illumination light source with significantlyreduced background light. Based on this design, each display scan framegenerates a frame of fingerprint signals and two sequential frames ofsignals are obtained by turning on the illumination light in one frameand off in the other frame. The subtraction of adjacent frames can beused to minimize or substantially reduce the ambient background lightinfluence. In implementations, the fingerprint sensing frame rate can beone half of the display frame rate.

In the example shown in FIG. 4B, a portion of the light from the one ormore illumination light sources 436 may also go through the cover topsurface and enter the finger tissues. This part of the illuminationlight is scattered around and a part of this scattered light may beeventually collected by the imaging sensor array in the opticalfingerprint sensor module 702. The light intensity of this scatteredlight is a result of interacting with the inner tissues of the fingerand thus depends on the finger's skin color, the blood concentration inthe finger tissue or the inner finger tissues. Such information of thefinger is carried by this scattered light on the finger, is useful forfingerprint sensing, and can be detected as part of the fingerprintsensing operation. For example, the intensity of a region of user'sfinger image can be integrated in detection for measuring or observingin increase or decrease in the blood concentration that is associatedwith or depends on the phase of the user's heart-beat. This signaturecan be used to determine the user's heart beat rate, to determine if theuser's finger is a live finger, or to provide a spoof device with afabricated fingerprint pattern. Additional examples of using informationin light carrying information on the inner tissues of a finger areprovided in later sections of this patent document.

The one or more illumination light sources 436 in FIG. 4B can bedesigned to emit illumination light of different colors or wavelengthsin some designs and the optical fingerprint sensor module can capturereturned light from a person's finger at the different colors orwavelengths. By recording the corresponding measured intensity of thereturned light at the different colors or wavelengths, informationassociated with the user's skin color, the blood flow or inner tissuestructures inside the finger can be measured or determined. As anexample, when a user registers a finger for fingerprint authenticationoperation, the optical fingerprint sensor can be operated to measure theintensity of the scatter light from the finger at two different colorsor illumination light wavelengths associated with light color A andlight color B, as intensities Ia and Ib, respectively. The ratio ofIa/Ib could be recorded to compare with later measurement when theuser's finger is placed on the sensing area on the top sensing surfaceto measure the fingerprint. This method can be used as part of thedevice's anti spoofing system to reject a spoof device that isfabricated with a fingerprint emulating or being identical to a user'sfingerprint but may not match user's skin color or other biologicalinformation of the user.

The one or more illumination light sources 436 can be controlled by thesame electronics 704 (e.g., MCU) for controlling the image sensor arrayin the block 702. The one or more illumination light sources 436 can bepulsed for a short time (e.g., at a low duty cycle) to emit lightintermittently and to provide pulse light for image sensing. The imagesensor array can be operated to monitor the light pattern at the samepulse duty cycle. If there is a human finger touching the sensing area615 on the screen, the image that is captured at the imaging sensingarray in the block 702 can be used to detect the touching event. Thecontrol electronics or MCU 704 connected to the image sensor array inthe block 702 can be operated to determine if the touch is by a humanfinger touch. If it is confirmed that it is a human finger touch event,the MCU 704 can be operated to wake up the smartphone system, turn onthe one or more illumination light sources 436 for performing theoptical fingerprint sensing), and use the normal mode to acquire a fullfingerprint image. The image sensor array in the block 702 sends theacquired fingerprint image to the smartphone main processor 705 whichcan be operated to match the captured fingerprint image to theregistered fingerprint database. If there is a match, the smartphoneunlocks the phone to allow a user to access the phone and start thenormal operation. If the captured image is not matched, the smartphoneproduces a feedback to user that the authentication is failed andmaintains the locking status of the phone. The user may try to gothrough the fingerprint sensing again, or may input a passcode as analternative way to unlock the phone.

In the example illustrated in FIGS. 4A and 4B, the under-screen opticalfingerprint sensor module uses the optically transparent block 701 andthe imaging sensing block 702 with the photodetector sensing array tooptically image the fingerprint pattern of a touching finger in contactwith the top surface of the display screen onto the photodetectorsensing array. The optical imaging axis or detection axis 625 from thesensing zone 615 to the photodetector array in the block 702 isillustrated in FIG. 4B for the illustrated example. The opticallytransparent block 701 and the front end of the imaging sensing block 702before the photodetector sensing array forma a bulk imaging module toachieve proper imaging for the optical fingerprint sensing. Due to theoptical distortions in this imaging process, a distortion correction canbe used to achieve the desired imaging operation.

In the optical sensing by the under-screen optical fingerprint sensormodule in FIGS. 4A and 4B and other designs disclosed herein, theoptical signal from the sensing zone 615 on the top transparent layer431 to the under-screen optical fingerprint sensor module includedifferent light components. FIGS. 5A-5C illustrate signal generation forthe returned light from the sensing zone 615 under different opticalconditions to facilitate the understanding of the operation of theunder-screen optical fingerprint sensor module. The light that entersinto the finger, either from the illumination light source or from otherlight sources (e.g., background light) can generate internally scatteredlight in tissues below the finger surface, such as the scattered light191 in FIGS. 5A-5C. Such internally scattered light in tissues below thefinger surface can propagate through the internal tissues of the fingerand subsequently transmits through the finger skin to enter the toptransparent layer 431 carrying certain information is not carried bylight that is scattered, refracted or reflected by the finger surface,e.g., information on finger skin color, the blood concentration or flowcharacteristics inside the finger, or an optical transmissive pattern ofthe finger that contains both (1) a two-dimensional spatial pattern ofexternal ridges and valleys of a fingerprint (2) an internal fingerprintpattern associated with internal finger tissue structures that give riseto the external ridges and valleys of a finger.

FIG. 5A shows an example of how illumination light from the one or moreillumination light sources 436 propagates through the OLED displaymodule 433, after transmitting through the top transparent layer 431,and generates different returned light signals including light signalsthat carry fingerprint pattern information to the under-screen opticalfingerprint sensor module. For simplicity, two illumination rays 80 and82 at two different locations are directed to the top transparent layer431 without experiencing total reflection at the interfaces of the toptransparent layer 431. Specifically, the illumination light rays 80 and82 are perpendicular or nearly perpendicular to the top layer 431. Afinger 60 is in contact with the sensing zone 615 on the e toptransparent layer 431. As illustrated, the illumination light beam 80reaches to a finger ridge in contact with the top transparent layer 431after transmitting through the top transparent layer 431 to generate thelight beam 183 in the finger tissue and another light beam 181 backtowards the LCD display module 433. The illumination light beam 82reaches to a finger valley located above the top transparent layer 431after transmitting through the top transparent layer 431 to generate thereflected light beam 185 from the interface with the top transparentlayer 431 back towards the LCD display module 433, a second light beam189 that enters the finger tissue and a third light beam 187 reflectedby the finger valley.

In the example in FIG. 5A, it is assumed that the finger skin'sequivalent index of refraction is about 1.44 at 550 nm and the coverglass index of refraction is about 1.51 for the top transparent layer431. The finger ridge-cover glass interface reflects part of the beam 80as reflected light 181 to bottom layers 524 below the LCD display module433. The reflectance can be low, e.g., about 0.1% in some LCD panels.The majority of the light beam 80 becomes the beam 183 that transmitsinto the finger tissue 60 which causes scattering of the light 183 toproduce the returned scattered light 191 towards the LCD display module433 and the bottom layers 524. The scattering of the transmitted lightbeam 189 from the LCD pixel 73 in the finger tissue also contributes tothe returned scattered light 191.

The beam 82 at the finger skin valley location 63 is reflected by thecover glass surface. In some designs, for example, the reflection may beabout 3.5% as the reflected light 185 towards bottom layers 524, and thefinger valley surface may reflect about 3.3% of the incident light power(light 187) to bottom layers 524 so that the total reflection may beabout 6.8%. The majority light 189 is transmitted into the fingertissues 60. Part of the light power in the transmitted light 189 in thefigure tissue is scattered by the tissue to contribute to the scatteredlight 191 towards and into the bottom layers 524.

Therefore, in the example in FIG. 5A, the light reflections from variousinterface or surfaces at finger valleys and finger ridges of a touchingfinger are different and the reflection ratio difference carries thefingerprint map information and can be measured to extract thefingerprint pattern of the portion that is in contact with the toptransparent layer 431 and is illuminated the OLED light.

FIGS. 5B and 5C illustrate optical paths of two additional types ofillumination light rays at the top surface under different conditionsand at different positions relative to valleys or ridges of a finger,including under a total reflection condition at the interface with thetop transparent layer 431. The illustrated illumination light raysgenerate different returned light signals including light signals thatcarry fingerprint pattern information to the under-screen opticalfingerprint sensor module. It is assumed that the cover glass 431 andthe LCD display module 433 are glued together without any air gap inbetween so that illumination light with a large incident angle to thecover glass 431 will be totally reflected at the cover glass-airinterface. FIGS. 5A, 5B and 5C illustrate examples of three differentgroups divergent light beams: (1) central beams 82 with small incidentangles to the cover glass 431 without the total reflection (FIG. 5A),(2) high contrast beams 201, 202, 211, 212 that are totally reflected atthe cover glass 431 when nothing touches the cover glass surface and canbe coupled into finger tissues when a finger touches the cover glass 431(FIGS. 5B and 5C), and (3) escaping beams having very large incidentangles that are totally reflected at the cover glass 431 even at alocation where the finger issue is in contact.

For the central light beams 82, the cover glass surface in some designsmay reflect about 0.1%˜3.5% to light beam 185 that is transmitted intobottom layers 524, the finger skin may reflect about 0.1%˜3.3% to lightbeam 187 that is also transmitted into bottom layers 524. The reflectiondifference is dependent on whether the light beams 82 meet with fingerskin ridge 61 or valley 63. The rest light beam 189 is coupled into thefinger tissues 60.

For high contrast light beams 201 and 202 meeting the local totallyinternal reflection condition, the cover glass surface reflects nearly100% to light beams 205 and 206 respectively if nothing touches thecover glass surface. When the finger skin ridges touch the cover glasssurface and at light beams 201 and 202 positions, most of the lightpower may be coupled into the finger tissues 60 by light beams 203 and204.

For high contrast light beams 211 and 212 meeting the local totallyinternal reflection condition, the cover glass surface reflects nearly100% to light beams 213 and 214 respectively if nothing touches thecover glass surface. When the finger touches the cover glass surface andthe finger skin valleys happen to be at light beams 211 and 212positions, no light power is coupled into finger tissues 60.

As illustrated in FIG. 5A, a portion of the illumination light that iscoupled into finger tissues 60 tends to experience random scattering bythe inner finger tissues to form low-contrast light 191 and part of suchlow-contrast light 191 can pass through the LCD display module 433 toreach to the optical fingerprint sensor module. This portion of lightcaptured by optical fingerprint sensor module contains additionalinformation on the finger skin color, blood characteristics and thefinger inner tissue structures associated with the fingerprint.Additional features for using internally scattered light in tissuesbelow the finger surface in optical sensing will be explained in laterpart of this patent document, such as obtaining an optical transmissivepattern of the finger that contains both (1) a two-dimensional spatialpattern of external ridges and valleys of a fingerprint (2) an internalfingerprint pattern associated with internal finger tissue structuresthat give rise to the external ridges and valleys of a finger.

Therefore, in high contrast light beams illuminated area, finger skinridges and valleys cause different optical reflections and thereflection difference pattern carries the fingerprint patterninformation. The high contrast fingerprint signals can be achieved bycomparing the difference.

The disclosed under-screen optical sensing technology can be in variousconfigurations to optically capture fingerprints based on the designillustrated in FIGS. 2A and 2B.

For example, the specific implementation in FIG. 4B based on opticalimaging by using a bulk imaging module in the optical sensing module canbe implemented in various configurations. FIGS. 6A-6C, 7, 8A-8B, 9,10A-10B, 11, and 12 illustrate examples of various implementations andadditional features and operations of the under-screen opticalfingerprint sensor module designs for optical fingerprint sensing.

FIGS. 6A-6C show an example of an under-screen optical fingerprintsensor module based on optical imaging via a lens for capturing afingerprint from a finger 445 pressing on the display cover glass 423.FIG. 6C is an enlarged view of the optical fingerprint sensor modulepart shown in FIG. 6B. The under-screen optical fingerprint sensormodule as shown in FIG. 6B is placed under the LCD display module 433includes an optically transparent spacer 617 that is engaged to thebottom surface of the LCD display module 433 to receive the returnedlight from the sensing zone 615 on the top surface of the toptransparent layer 431, an imaging lens 621 that is located between andspacer 617 and the photodetector array 623 to image the receivedreturned light from the sensing zone 615 onto the photodetector array623. Different from FIG. 4B showing an example of an optical projectionimaging system without a lens, the example of the imaging design in FIG.6B used the imaging lens 621 to capture the fingerprint image at thephotodetector array 623 and enables an image reduction by the design ofthe imaging lens 621. Similar to the imaging system in the example inFIG. 4B to some extent, this imaging system in FIG. 6B for the opticalfingerprint sensor module can experience image distortions and asuitable optical correction calibration can be used to reduce suchdistortions, e.g., the distortion correction methods described for thesystem in FIG. 4B.

Similar to the assumptions in FIGS. 5A-5C, it is assumed that the fingerskin's equivalent index of refraction to be about 1.44 at 550 nm and abare cover glass index of refraction to be about 1.51 for the coverglass 423. When the OLED display module 433 is glued onto the coverglass 431 without any air gap, the total inner reflection happens inlarge angles at or larger than the critical incident angle for theinterface. The total reflection incident angle is about 41.8° if nothingis in contact with the cover glass top surface, and the total reflectionangle is about 73.7° if the finger skin touches the cover glass topsurface. The corresponding total reflection angle difference is about31.9°.

In this design, the micro lens 621 and the photodiode array 623 define aviewing angle θ for capturing the image of a contact finger in thesensing zone 615. This viewing angle can be aligned properly bycontrolling the physical parameters or configurations in order to detecta desired part of the cover glass surface in the sensing zone 615. Forexample, the viewing angle may be aligned to detect the total innerreflection of the LCD display assembly. Specifically, the viewing angleθ is aligned to sense the effective sensing zone 615 on the cover glasssurface. The effective sensing cover glass surface 615 may be viewed asa mirror so that the photodetector array effectively detects an image ofthe fingerprint illumination light zone 613 in the LCD display that isprojected by the sensing cover glass surface 615 onto the photodetectorarray. The photodiode/photodetector array 623 can receive the image ofthe zone 613 that is reflected by the sensing cover glass surface 615.When a finger touches the sensing zone 615, some of the light can becoupled into the fingerprint's ridges and this will cause thephotodetector array to receive light from the location of the ridges toappear as a darker image of the fingerprint. Because the geometrics ofthe optical detection path are known, the fingerprint image distortioncaused in the optical path in the optical fingerprint sensor module canbe corrected.

Consider, as a specific example, that the distance H in FIG. 6B from thedetection module central axis to the cover glass top surface is 2 mm.This design can directly cover 5 mm of an effective sensing zone 615with a width Wc on the cover glass. Adjusting the spacer 617 thicknesscan adjust the detector position parameter H, and the effective sensingzone width Wc can be optimized. Because H includes the thickness of thecover glass 431 and the display module 433, the application designshould take these layers into account. The spacer 617, the micro lens621, and the photodiode array 623 can be integrated under the colorcoating 619 on the bottom surface of the top transparent layer 431.

FIG. 7 shows an example of further design considerations of the opticalimaging design for the optical fingerprint sensor module shown in FIGS.6A-6C by using a special spacer 618 to replace the spacer 617 in FIGS.6B-6C to increase the size of the sensing area 615. The spacer 618 isdesigned with a width Ws and thickness is Hs to have a low refractionindex (RI) ns, and is placed under the LCD display module 433, e.g.,being attached (e.g., glued) to the bottom surface the LCD displaymodule 433. The end facet of the spacer 618 is an angled or slantedfacet that interfaces with the micro lens 621. This relative position ofthe spacer and the lens is different from FIGS. 6B-6C where the lens isplaced underneath the spacer 617. The micro lens 621 and a photodiodearray 623 are assembled into the optical detection module with adetection angle width 0. The detection axis 625 is bent due to opticalrefraction at the interface between the spacer 618 and display module433 and at the interface between the cover glass 431 and the air. Thelocal incident angle ϕ1 and ϕ2 are decided by the refractive indicesRIs, ns, nc, and na of the materials for the components.

If nc is greater than ns, ϕ1 is greater than ϕ2. Thus, the refractionenlarges the sensing width Wc. For example, assuming the finger skin'sequivalent RI is about 1.44 at 550 nm and the cover glass index RI isabout 1.51, the total reflection incident angle is estimated to be about41.8° if nothing touches the cover glass top surface, and the totalreflection angle is about 73.7° if the finger skin touches the coverglass top surface. The corresponding total reflection angle differenceis about 31.9°. If the spacer 618 is made of same material of the coverglass, and the distance from the detection module center to the coverglass top surface is 2 mm, if detection angle width is θ=31.9°, theeffective sensing area width Wc is about 5 mm. The corresponding centralaxis's local incident angle is ϕ1=ϕ2=57.75°. If the material for thespecial spacer 618 has a refractive index ns about 1.4, and Hs is 1.2 mmand the detection module is tilted at ϕ1=70°. The effective sensing areawidth is increased to be greater than 6.5 mm. Under those parameters,the detection angle width in the cover glass is reduced to 19°.Therefore, the imaging system for the optical fingerprint sensor modulecan be designed to desirably enlarge the size of the sensing area 615 onthe top transparent layer 431.

the refractive index RI of the special spacer 618 is designed to besufficiently low (e.g., to use MgF₂, CaF₂, or even air to form thespacer), the width Wc of the effective sensing area 615 is no longerlimited by the thickness of the cover glass 431 and the display module433. This property provides desired design flexibility. In principle, ifthe detection module has a sufficient resolution, the effective sensingarea may even be increased to cover the entire display screen.

Since the disclosed optical sensor technology can be used to provide alarge sensing area for capturing a pattern, the disclosed under-screenoptical fingerprint sensor modules may be used to capture and detect notonly a pattern of a finger but a larger size patter such a person's palmthat is associated with a person for user authentication.

FIGS. 8A-8B show an example of further design considerations of theoptical imaging design for the optical fingerprint sensor module shownin FIG. 7 by setting the detection angle θ′ of the photodetector arrayrelative in the display screen surface and the distance L between thelens 621 and the spacer 618. FIG. 8A shows a cross-sectional view alongthe direction perpendicular to the display screen surface, and FIG. 8Bshows a view of the device from either the bottom or top of the displacescreen. A filling material 618 c can be used to fill the space betweenthe lens 621 and the photodetector array 623. For example, the fillingmaterial 618 c can be same material of the special spacer 618 or anotherdifferent material. In some designs, the filling material 618 c may theair space.

FIG. 9 shows another example of an under-screen optical fingerprintsensor module based on the design in FIG. 7 where one or moreillumination light sources 614 are provided to illuminate the topsurface sensing zone 615 for optical fingerprint sensing. Theillumination light sources 614 may be of an expanded type, or be acollimated type so that all the points within the effective sensing zone615 is illuminated. The illumination light sources 614 may be a singleelement light source or an array of light sources.

FIGS. 10A-10B show an example of an under-screen optical fingerprintsensor module that uses an optical coupler 628 shaped as a thin wedge toimprove the optical detection at the optical sensor array 623. FIG. 10Ashows a cross section of the device structure with an under-screenoptical fingerprint sensor module for fingerprint sensing and FIG. 10Bshows a top view of the device screen. The optical wedge 628 (with arefractive index ns) is located below the display panel structure tomodify a total reflection condition on a bottom surface of the displaypanel structure that interfaces with the optical wedge 628 to permitextraction of light out of the display panel structure through thebottom surface. The optical sensor array 623 receives the light from theoptical wedge 628 extracted from the display panel structure and theoptical imaging module 621 is located between the optical wedge 628 andthe optical sensor array 623 to image the light from the optical wedge628 onto the optical sensor array 623. In the illustrated example, theoptical wedge 628 includes a slanted optical wedge surface facing theoptical imaging module and the optical sensing array 623. Also, asshown, there is a free space between the optical wedge 628 and theoptical imaging module 621.

If the light is totally reflected at the sensing surface of the coverglass 431, the reflectance is 100%, of the highest efficiency. However,the light will also be totally reflected at the LCD bottom surface 433 bif it is parallel to the cover glass surfaces. The wedge coupler 628 isused to modify the local surface angle so that the light can be coupledout for the detection at the optical sensor array 623. The micro holesin the LCD display module 433 provide the desired light propagation pathfor light to transmit through the LCD display module 433 for theunder-screen optical sensing. The actual light transmission efficiencymay gradually be reduced if the light transmission angle becomes toolarge or when the TFT layer becomes too thick. When the angle is closeto the total reflection angle, namely about 41.8° when the cover glassrefractive index is 1.5, the fingerprint image looks good. Accordingly,the wedge angle of the wedge coupler 628 may be adjusted to be of acouple of degrees so that the detection efficiency can be increased oroptimized. If the cover glass' refractive index is selected to behigher, the total reflection angle becomes smaller. For example, if thecover glass is made of Sapphire which refractive index is about 1.76,the total reflection angle is about 34.62°. The detection lighttransmission efficiency in the display is also improved. Therefore, thisdesign of using a thin wedge to set the detection angle to be higherthan the total reflection angle, and/or to use high refractive indexcover glass material to improve the detection efficiency.

In the under-screen optical fingerprint sensor module designs in FIGS.6A-6C, 7, 8A-8B, 9, and 10A-10B, the sensing area 615 on the toptransparent surface is not vertical or perpendicular to the detectionaxis 625 of the optical fingerprint sensor module so that the imageplane of the sensing area is also not vertical or perpendicular to thedetection axis 625. Accordingly, the plane of the photodetector array523 can be tilted relative the detection axis 625 to achieve highquality imaging at the photodetector array 623.

FIG. 11 shows three example configurations for this tilting. FIG. 11 (1)shows the sensing area 615 a is tilted and is not perpendicular thedetection axis 625. In a specified case shown in (2), the sensing area615 b is aligned to be on the detection axis 625, its image plane willalso be located on the detection axis 625. In practice, the lens 621 canbe partially cut off so as to simplify the package. In variousimplementations, the micro lens 621 can also be of transmission type orreflection type. For example, a specified approach is illustrated in(3). The sensing area 615 c is imaged by an imaging mirror 621 a. Aphotodiode array 623 b is aligned to detect the signals.

In the above designs where the lens 621 is used, the lens 621 can bedesigned to have an effective aperture that is larger than the apertureof the holes in the LCD display layers that allow transmission of lightthrough the LCD display module for optical fingerprint sensing. Thisdesign can reduce the undesired influence of the wiring structures andother scattering objects in the LCD display module.

FIG. 12 shows an example of an operation of the fingerprint sensor forreducing or eliminating undesired contributions from the backgroundlight in fingerprint sensing. The optical sensor array can be used tocapture various frames and the captured frames can be used to performdifferential and averaging operations among multiple frames to reducethe influence of the background light. For example, in frame A, theillumination light source for optical fingerprint sensing is turned onto illuminate the finger touching area, in frame B the illumination ischanged or is turned off. Subtraction of the signals of frame B from thesignals of frame A can be used in the image processing to reduce theundesired background light influence.

The undesired background light in the fingerprint sensing may also bereduced by providing proper optical filtering in the light path. One ormore optical filters may be used to reject the environment lightwavelengths, such as near IR and partial of the red light etc. In someimplementation, such optical filter coatings may be made on the surfacesof the optical parts, including the display bottom surface, prismsurfaces, sensor surface etc. For example, human fingers absorb most ofthe energy of the wavelengths under ˜580 nm, if one or more opticalfilters or optical filtering coatings can be designed to reject light inwavelengths from 580 nm to infrared, undesired contributions to theoptical detection in fingerprint sensing from the environment light maybe greatly reduced.

FIG. 13 shows an example of an operation process for correcting theimage distortion in the optical fingerprint sensor module. At step 1301,the one or more illumination light sources are controlled and operatedto emit light in a specific region, and the light emission of suchpixels is modulated by a frequency F. Ate step 1302, an imaging sensorunder the display panel is operated to capture the image at frame rateat same frequency F. In the optical fingerprint sensing operation, afinger is placed on top of the display panel cover substrate and thepresence of the finger modulates the light reflection intensity of thedisplay panel cover substrate top surface. The imaging sensor under thedisplay captures the fingerprint modulated reflection light pattern. Atstep 1303, the demodulation of the signals from image sensors issynchronized with the frequency F, and the background subtraction isperformed. The resultant image has a reduced background light effect andincludes images from pixel emitting lights. At step 1304, the captureimage is processed and calibrated to correct image system distortions.At step 1305, the corrected image is used as a human fingerprint imagefor user authentication.

The same optical sensors used for capturing the fingerprint of a usercan be used also to capture the scattered light from the illuminatedfinger as shown by the back scattered light 191 in FIG. 5A. The detectorsignals from the back scattered light 191 in FIG. 5A in a region ofinterest can be integrated to produce an intensity signal. The intensityvariation of this intensity signal is evaluated to determine otherparameters beyond the fingerprint pattern, e.g., the heart rate of theuser or inner topological tissues of a finger associated with theexternal fingerprint pattern.

The above fingerprint sensor may be hacked by malicious individuals whocan obtain the authorized user's fingerprint, and copy the stolenfingerprint pattern on a carrier object that resembles a human finger.Such unauthorized fingerprint patterns may be used on the fingerprintsensor to unlock the targeted device. Hence, a fingerprint pattern,although a unique biometric identifier, may not be by itself acompletely reliable or secure identification. The under-screen opticalfingerprint sensor module can also be used to as an opticalanti-spoofing sensor for sensing whether an input object withfingerprint patterns is a finger from a living person and fordetermining whether a fingerprint input is a fingerprint spoofingattack. This optical anti-spoofing sensing function can be providedwithout using a separate optical sensor. The optical anti-spoofing canprovide high-speed responses without compromising the overall responsespeed of the fingerprint sensing operation.

FIG. 14 shows exemplary optical extinction coefficients of materialsbeing monitored in blood where the optical absorptions are differentbetween the visible spectral range e.g., red light at 660 nm and theinfrared range, e.g., IR light at 940 nm. By using probe light toilluminate a finger at a first visible wavelength (Color A) and a seconddifferent wavelength such as an infrared (IR) wavelength (Color B), thedifferences in the optical absorption of the input object can becaptured determine whether the touched object is a finger from a liveperson. The one or more illumination light sources for providing theillumination for optical sensing can be used to emit light of differentcolors to emit probe or illumination light at least two differentoptical wavelengths to use the different optical absorption behaviors ofthe blood for live finger detection. When a person' heart beats, thepulse pressure pumps the blood to flow in the arteries, so theextinction ratio of the materials being monitored in the blood changeswith the pulse. The received signal carries the pulse signals. Theseproperties of the blood can be used to detect whether the monitoredmaterial is a live-fingerprint or a fake fingerprint.

FIG. 15 shows a comparison between optical signal behaviors in thereflected light from a nonliving material (e.g., a fake finger or aspoof device with a fabricated fingerprint pattern) and a live finger.The optical fingerprint sensor can also operate as a heartbeat sensor tomonitor a living organism. When two or more wavelengths of the probelight are detected, the extinction ratio difference can be used toquickly determine whether the monitored material is a living organism,such as live fingerprint. In the example shown in FIG. 15, probe lightat different wavelengths were used, one at a visible wavelength andanother at an IR wavelength as illustrated in FIG. 14.

When a nonliving material touches the top cover glass above thefingerprint sensor module, the received signal reveals strength levelsthat are correlated to the surface pattern of the nonliving material andthe received signal does not contain signal components associated with afinger of a living person. However, when a finger of a living persontouches the top cover glass, the received signal reveals signalcharacteristics associated with a living person, including obviouslydifferent strength levels because the extinction ratios are differentfor different wavelengths. This method does not take long time todetermine whether the touching material is a part of a living person. InFIG. 15, the pulse-shaped signal reflects multiple touches instead ofblood pulse. Similar multiple touches with a nonliving material does notshow the difference caused by a living finger.

This optical sensing of different optical absorption behaviors of theblood at different optical wavelengths can be performed in a shortperiod for live finger detection and can be faster than opticaldetection of a person's heart beat using the same optical sensor.

In LCD displays, the LCD backlighting illumination light is white lightand thus contains light at both the visible and IR spectral ranges forperforming the above live finger detection at the optical fingerprintsensor module. The LCD color filters in the LCD display module can beused to allow the optical fingerprint sensor module to obtainmeasurements in FIGS. 14 and 15. In addition, the designated lightsources 436 for producing the illumination light for optical sensing canbe operated to emit probe light at the selected visible wavelength andIR wavelength at different times and the reflected probe light at thetwo different wavelengths is captured by the optical detector array 623to determine whether touched object is a live finger based on the aboveoperations shown in FIGS. 14 and 15. Notably, although the reflectedprobe light at the selected visible wavelength and IR wavelength atdifferent times may reflect different optical absorption properties ofthe blood, the fingerprint image is always captured by both the probelight the selected visible wavelength and the probe light at the IRwavelength at different times. Therefore, the fingerprint sensing can bemade at both the visible wavelength and IR wavelength.

FIG. 16 shows an example of an operation process for determining whetheran object in contact with the LCD display screen is part of a finger ofa live person by operating the one or more illumination light sourcesfor optical sensing to illuminate the finger with light in two differentlight colors.

For yet another example, the disclosed optical sensor technology can beused to detect whether the captured or detected pattern of a fingerprintor palm is from a live person's hand by a “live finger” detectionmechanism by other mechanisms other than the above described differentoptical absorptions of blood at different optical wavelengths. Forexample, a live person's finger tends to be moving or stretching due tothe person's natural movement or motion (either intended or unintended)or pulsing when the blood flows through the person's body in connectionwith the heartbeat. In one implementation, the optical fingerprintsensor module can detect a change in the returned light from a finger orpalm due to the heartbeat/blood flow change and thus to detect whetherthere is a live heartbeat in the object presented as a finger or palm.The user authentication can be based on the combination of the both theoptical sensing of the fingerprint/palm pattern and the positivedetermination of the presence of a live person to enhance the accesscontrol. For yet another example, as a person touches the LCD displayscreen, a change in the touching force can be reflected in one or moreways, including fingerprint pattern deforming, a change in thecontacting area between the finger and the screen surface, fingerprintridge widening, or a change in the blood flow dynamics. Those and otherchanges can be measured by optical sensing based on the disclosedoptical sensor technology and can be used to calculate the touch force.This touch force sensing can be used to add more functions to theoptical fingerprint sensor module beyond the fingerprint sensing.

In the above examples where the fingerprint pattern is captured on theoptical sensor array via an imaging module as in FIG. 4B and FIG. 6B,optical distortions tend to degrade the image sensing fidelity. Suchoptical distortions can be corrected in various ways. For example, aknown pattern can be used to generate an optical image at the opticalsensor array and the image coordinates in the know pattern can becorrelated to the generated optical image with distortions at theoptical sensor array for calibrating the imaging sensing signals outputby the optical sensor array for fingerprint sensing. The fingerprintsensing module calibrates the output coordinates referencing on theimage of the standard pattern.

In light of the disclosure in this patent document, variousimplementations can be made for the optical fingerprint sensor module asdisclosed.

For example, a display panel can be constructed in which each pixelemitting lights, and can be controlled individually; the display panelincludes an at least partially transparent substrate; and a coversubstrate, which is substantially transparent. An optical fingerprintsensor module is placed under the display panel to sense the images formon the top of the display panel surface. The optical fingerprint sensormodule can be used to sense the images form from light emitting fromdisplay panel pixels. The optical fingerprint sensor module can includea transparent block with refractive index lower than the display panelsubstrate, and an imaging sensor block with an imaging sensor array andan optical imaging lens. In some implementations, the low refractiveindex block has refractive index in the range of 1.35 to 1.46 or 1 to1.35.

For another example, a method can be provided for fingerprint sensing,where light emitting from a display panel is reflected off the coversubstrate, a finger placed on top of the cover substrate interacts withthe light to modulate the light reflection pattern by the fingerprint.An imaging sensing module under the display panel is used to sense thereflected light pattern image and reconstruct fingerprint image. In oneimplementation, the emitting light from the display panel is modulatedin time domain, and the imaging sensor is synchronized with themodulation of the emitting pixels, where a demodulation process willreject most of the background light (light not from pixels beingtargeted).

III. Lens-Pinhole Imaging Designs for Under-Display Optical Sensing

In various implementations of the under-screen optical fingerprintsensor module technology for fingerprint sensing disclosed herein, animagine module having at least one imaging lens can be used to achievethe optical imaging of the illuminated touched portion of a finger ontothe optical sensor array in the under-screen optical fingerprint sensormodule. The lensing effect of the imaging module is in part forcontrolling the spatial spreading of the returned light that mayspatially scramble returned light from different locations on thetouched portion of the finger at the optical sensor array so that thespatial information on the returned light corresponding to thefingerprint pattern on a finger can be preserved by the imaging lenswith a desired spatial imaging resolution when the imaging lens directsthe returned light to reach the optical sensor array. The spatialimaging resolution of an imaging module having a single imaging lens oran assembly of two or more imaging lenses is proportional to thenumerical aperture of the imaging module. Accordingly, a high-resolutionimaging lens requires a large numerical aperture and thus a lens with alarge diameter. This aspect of a lens-based imaging module inevitablyrequires a bulking lens system to produce a high-resolution imagingsystem. In addition, a given imaging lens has a limited field of viewwhich increases as the focal length decreases and decreases as the focallength increases.

In many fingerprint sensing applications such as optical fingerprintsensors implemented under a display screen in a mobile device, it isdesirable to have a compact imaging system with a high spatial imagingresolution and a large field of view. In view of the trade-offs invarious imaging features of a lens-based imaging system discussed above,a compact optical imaging system for optical fingerprint sensing isprovided below by combining a lens and a pinhole as a lens-pinholeimaging system where the lens is used to form a lens-based imagingsystem to achieve a high spatial imaging resolution via the lens and areduced size in the captured image at the optical detector array toreduce the size the optical detector array via the same lens and thepinhole is placed in front of the lens to produce a large field of viewin optical imaging by effectuating a pinhole camera without requiring alens of a large diameter. A conventional pinhole camera can include asmall aperture for optical imaging and can produce a large field of viewwhile suffering a limited image brightness due to the small aperture anda low spatial imaging resolution. A combination of an imaging lens and apinhole camera, when properly designed, can benefit from the highspatial imaging resolution of the imaging lens and the large field ofview of the pinhole camera.

FIG. 17 shows one example of an optical fingerprint sensor module 4620placed under an LCD display screen where a pinhole and a lens are usedto form the optical imaging system for the optical fingerprint sensormodule 4620. One or more illumination light sources 436 are provided ata location under the LCD display module 433 to produce illuminationlight to pass through the LCD display module 433 and the top transparentlayer 431 which includes a sensing zone 615 on the top surface of thetransparent layer 431. One or more illumination light sources 4661 areprovided under the top transparent layer 431 provided to produceillumination light to sensing zone 615 on the top surface of thetransparent layer 431. The one or more illumination light sources 436may be located in or next to the optical fingerprint sensor module 4620to provide, in addition to providing illumination for fingerprintsensing, a breathing light indicator to indicate that the opticalfingerprint sensing is in progress or the optical fingerprint sensormodule 4620 is turned on or activated.

In the illustrated example in FIG. 17, the optical sensing module 4620is a compact module by using a micro lens 4621 e with a small diameterthat can be about the same size of the pinhole so slightly larger thanthe pinhole. The micro lens 4621 e is engaged to a pinhole structure4621 g that is optically opaque and may be a layer of a blackened ormetal material formed on a surface of a pinhole substrate 4621 f of anoptically transparent material with an opening as the pinhole 4643. Themicro lens 4621 e is placed on the lower side of the pinhole substrate4621 f. In operation, the optical layers above the pinhole 4643 in thepinhole structure 4621 g are structured to produce a large optical fieldof view in collecting the returned light from the LCD display panel andto transmit the collected light towards the optical sensor array 4623 e.The optical detectors in the optical sensor array 4623 e respond to thereceived optical pattern to produce detector signals and a detectorcircuitry module 4623 f is coupled to the optical sensor array 4623 e toreceive and process the detectors signals. The detector circuitry module4623 f may include, in some implementations, a flexible printed circuit(PFC). The micro lens 4621 e receives the transmitted light from thepinhole and to focus the received light onto the optical sensor array4623 e for optical imaging at an enhanced spatial imaging resolution atthe optical sensor array 4623 e when compared to a lower spatial imagingresolution of the pinhole in projecting light onto the optical sensorarray 623 e without the micro lens 4621 e. In this design, the lowresolution of the pinhole is compensated by using the micro lens 4621 eand the limited field of view of the micro lens 4621 e is compensated bythe large field of view of the assembly of the low-index layer 4618 e, ahigh-index layer 4621 f and the pinhole 4643.

In the illustrated example of using the pinhole-lens assembly foroptical imaging in FIG. 17, the object plane of the pinhole-lensassembly is near the top effective sensing zone 615 on the top surfaceof the transparent layer 4431 such as a cover glass for the touchsensing LCD display panel and the imaging plane of the pinhole-lensassembly is the receiving surface of the optical detectors of theoptical sensor array 4623 e. In addition to the pinhole substrate 4621f, an optically transparent spacer 4618 e with a refractive index lowerthan that of the pinhole substrate 4621 f is provided between thepinhole substrate 621 f and the LCD display panel. This use of a lowerindex material above the pinhole substrate 4621 f is part of the opticaldesign to achieve a large field of view for receiving light from the LCDdisplay panel. In some implementations, the lower-index spacer 4618 emay be an air gap. This design provides an optical interface of twodifferent optical materials between lower-index spacer 4618 e and thehigher-index pinhole substrate 4621 f and the optical refraction at thisinterface converts a large field of view (FOV) (e.g., around 140 degreein some cases) of incident light from the LCD display panel in thelower-index spacer 4618 e into a smaller FOV in the higher-index pinholesubstrate 4621 f Accordingly, the output light rays produced by thepinhole-lens assembly have a relatively small FOV.

This design of reducing the FOV can be advantageous in several aspects.First, the optical input FOV in the lower-index spacer 4618 e of theoptical fingerprint sensor module 4620 allows the input to have a largeFOV. Second, the actual FOV handled at by the pinhole-lens assemblylocated below the higher-index pinhole substrate 4621 f is a reduced FOVwith respect to the optical input FOV so that light rays with largeincident angles are limited by this reduced FOV. This is beneficialbecause image distortions caused by light rays at large incident anglesat the pinhole-lens assembly are reduced by this reduced FOV. Inaddition, this reduced FOV at the pinhole-lens assembly reduces theundesired pinhole shading effect that would distort the brightnessdistribution of the image at the optical sensor array.

Different from a convention pinhole camera with uses a pinhole with adiameter around 40 microns in some pinhole camera designs, the pinhole4643 formed in the opaque layer 4621 g is designed to have a diametermuch larger than the typical size of a pinhole in a pinhole camera,e.g., greater than 100 microns, or 200 microns (e.g., 250 microns) insome designs. In this combination of the lens and the pinhole, the useof the high-index material for the pinhole substrate 4612 f just abovethe pinhole 4643 and the use of the lower-index layer 4618 e above thepinhole substrate 4612 f allows the pinhole 4643 to have a diameter muchlarger than the typical size of a pinhole in a pinhole camera whilestill achieving a large FOV. For example, in some implementations, thediameter of the pinhole 4643 may be about the same as or similar to theradius of curvature of the curve surface of the lens 4621 e whenstructured as a half ball lens with a flat surface facing the pinhole4643 and a partial spherical surface that directs the light from thepinhole 643 towards the photodetector array 4621 e.

Additional design features can also be implemented to improve theoverall optical performance and the compactness of the optical imagingsystem based on the pinhole-lens assembly. For example, as illustratedin FIG. 17, additional optical layers can be placed between thelens-pinhole assembly and the photodiode array 4623 e. In this example,an optically transparent spacer 4621 h and a protection layer 4623 g areprovided in the light path from the pinhole-lens assembly to the opticalsensor array 4623 e. In some implementations, the spacer 4621 h may be alow-index layer such as an air gap, and the protection layer 4623 g maybe a layer covering the top of the optical detectors of the opticalsensor array 4623 e and having a refractive index higher than that ofthe spacer 4621 h. The layers 4621 h and 4623 g can be structured toreduce or eliminate the imaging distortion at the optical sensor array4623 e. When light is refracted at media interfaces, the nonlinearity inthe directions of refracted rays exists and creates image distortions atthe optical sensor array 4623 e. Such distortions become more pronouncedwhen the incident angles are large. To reduce such distortions, theoptical thickness ratio of spacer 4621 h and 4623 g can be selected inlight of the optical structure of the pinhole-lens assembly and theoptical objective field of the pinhole-lens assembly (e.g., the opticallayers from the top sensing surface of the top glass layer 4431 to thepinhole substrate 4621 f).

Optical distortions occur at each interface of different opticalmaterials along the optical path of light from the top of the LCDdisplay panel to the optical sensor array 4623 e. One design techniquefor reducing such optical distortions is to provide optically matchingstructures on lower side of the pinhole-lens assembly (i.e., the opticallayers on the imaging side of the pinhole-lens assembly) tocorresponding to optical structures on the upper side of thepinhole-lens assembly (i.e., the optical layers on the object side ofthe pinhole-lens assembly) so that an optical distortion incurred at oneinterface along the optical path from the LCD panel to the pinhole-lensassembly in the object side of the pinhole-lens assembly is countered oroffset by optical refraction at a matching interface along the opticalpath from the pinhole-lens assembly to the optical sensor array 4623 ein the imaging side of the pinhole-lens assembly. The optical matchinglayers in the imaging side of the pinhole-lens assembly are designed bytaking into account of the optical power of the lens in the pinhole-lensassembly. In a pinhole imaging system with the pinhole 4643 alonewithout the lens 4621 e, optical distortions are present when the mediaare not matched between the object and the image fields. Such opticaldistortions may be in form of a barrel distortion when the FOV is largeby using a grid pattern placed over the top sensing surface to test thedistortions. The barrel distortions caused by the un-matched opticallayers between the object and the image fields of the pinhole 4643 isundesirable because they directly impact the accuracy of the fingerprintpattern captured by the optical sensor array 4623 e. It is noted thatthe level of such distortions is usually higher in the central part ofthe imaging field at the optical sensor array 4623 e than the peripheralpart.

To mitigate such distortions, material layers below the pinhole 4643 andthe lens 4621 e in the imaging field can be structured in terms of theirrefractive indices and thickness values to reverse the distortionsintroduced by the material layers in the object side. This is achievedby matching the refraction behavior at large incident angles so as tocorrect the image to be linearly formed on the detector surface. Forexample, in a pinhole imaging system with an imaging magnification at ⅕,if there are a glass layer of 2 mm thick and an air gap layer of 1 mmthick above the pinhole 4643, a glass layer of 0.4 mm thick and an airgap of 0.25 mm thick can be provided below the pinhole 4643 and abovethe optical sensor array 4623 e to reduce the optical distortions at theoptical sensor array 4623 e. This technique can be applied to providematching layers below the pinhole 4643 for complex material layers abovethe pinhole 4643.

FIG. 18 shows an example of different layers in the LCD display moduleabove the optical fingerprint sensor module. In this embodiment, thesensor module 4620 is integrated under the LCD display module.Illumination light sources 4661 and extra light sources 4664 are alsointegrated in or close to the sensor module 4620. The virtual button forthe optical fingerprint (FP) sensing shown within the LCD display screennear the edge is virtual because it is on an area of the contiguous topsurface across the entire display (there is no separate physical buttonfor fingerprint sensing) and is a displayed area that is indicated asthe effective sensing zone 615 (FIG. 17) for fingerprint sensing.

In the example in FIG. 18, the materials in the LCD backlightingcomponents of the LCD display module are designed to provide opticaltransmission paths to allow returned probe light from the top sensingzone to reach the underlying optical fingerprint sensor module below theLCD display module. Light transmitting holes or slits can be formed insuch materials. To modify the appearance of the display at the opticalsensor position to make the optical sensor less visible, a diffuser film4433 d may be integrated with the prisms 4433 c above the backlightingwaveguide layer 4433 f to diffuse the light towards the LCD pixels foruniform illumination, and additional diffuser 4433 i may be integratedwith the mirror film 4433 g that is below the backlighting waveguidelayer 4433 f and reflects or recycles backlighting light back to the LCDlayers to improve the illumination efficiency. In some implementations,the additional diffuser films 4433 d and 4433 i may be shaped as stripesaround edges around the optical fingerprint sensor module to diffuse thelight scattered in the light paths so that the light path edges aroundthe optical fingerprint sensor module under the LCD display module arehidden and or less visible.

FIG. 19 shows an example implementation of the design in FIG. 17 wherethe pinhole layer coating 4621 g on the bottom surface of the pinholesubstrate 4621 f is structured to have a high optical reflection at itsupper side facing the pinhole substrate 4621 f. This feature is used tocompensate the regional reflection of the mirror film hole or theoptical reflector layer or diffusion layer in the LCD layers above theoptical fingerprint sensor module 4620.

In some implementations, an additional diffuser layer 4621 i may beapplied on top of the pinhole substrate 4621 f as shown in FIG. 19 tocause optical diffusion and the central part of the additional diffuserlayer 4621 i can include a clear light path to receive the returnedlight from the LCD display module for optical sensing.

Furthermore, FIG. 19 shows an example location of one or more extralight sources 436 a installed under the additional diffuser layer 4621i. The light emitted from the extra light sources 436 a is scattered bythe diffuser 4621 i. The extra light sources 436 a may emit light ofdifferent optical wavelengths, e.g., including wavelengths for fingerillumination and other wavelengths for other functions, such asbreathing light function etc.

FIG. 20 shows an example where one or more under cover glass lightsources 4661 are provided to illuminate the touching finger 4447 foroptical sensing. The light produced by one or more under cover glasslight sources 4661 is represented by light 4661 a and can transmit intothe finger tissues or propagate through the corneum of the finger skin.For example, when light 4661 a transmits through the finger skin andpenetrates into the finger tissues, the light is scattered by the fingertissues. A part of the scattered light comes out as the signal light4661 b carrying the fingerprint information including information ontopological inner tissue structures associated with the externalfingerprint pattern and is detected by the optical fingerprint sensormodule 4620. The signal light 4661 b is incident to the sensing zone onthe top glass at a large angle can capture and carry the fingerprintinformation when the skin is wet or dry. Hence, using the one or moreunder cover glass light sources 4661 can improve the sensor's detectionperformance and reliability.

The pinhole-lens assembly for optical imaging in the example in FIG. 17can achieve a higher spatial imaging resolution to capture fine featuresin the captured images beyond the spatial imaging resolution of thesystem with the pinhole 4643 alone without the lens 4621 e. This higherspatial imaging resolution is a result of having the lens 4621 e. FIGS.21A and 21B illustrates the imaging operation of the pinhole alone andthe imaging operation of the pinhole-lens assembly.

Referring to FIG. 21A showing a pinhole imaging system without the lens,the pinhole 4643 diffracts the incident light beam 4661 to produce adiffracted the output light beam 4673 that is divergent due to thediffraction by the pinhole 4643. This divergent light beam 4673 forms animage light spot 4679 at the imaging plane 4667 that reflects theresolution of this imaging system.

FIG. 21B shows a micro lens 4621 e is added under the pinhole 4643. As aresult of this combining the micro lens 4621 e and the pinhole 4643, thecurvature of the micro lens 4621 e modifies the wave-front of the lightbeam diffracted by the pinhole 4643 to produce a light spot 4681 at theimaging plane 4667 which is smaller than the light spot 4679 produced bythe pinhole 4643 alone without the lens 4621 e.

The pinhole-lens assembly can be implemented to provide a compactoptical fingerprint sensor module 4620 in the example in FIG. 17. Due tothe refraction at the media interfaces, the light propagation angle canbe controlled by using different optical materials. For example, asshown in FIG. 22, if the refractive index n1 in the media above thepinhole substrate 4621 f is lower than the refractive index n2 of thepinhole substrate 4621 f, a light beam 4683 with a large incident angleis bent to a beam 4685 with a smaller angle after entering the pinholesubstrate 4621 f. Therefore, an extremely large field of view can berealized for receiving input light at the object side of thepinhole-lens assembly by using a higher index material for the pinholesubstrate 4621 f. In some implementations, a large FOV (e.g., close toor above 140 degrees) may be achieved by using a high-index material forthe pinhole substrate 4621 f to create a sufficiently large differencebetween the refractive indices the pinhole substrate 621 f and the layerabove the pinhole substrate 4621 f.

The above design for achieving a large diffraction bending of light raysat the top surface of the pinhole substrate 4621 f can be used to reducethe thickness of the optical fingerprint sensor module by incorporatingsome low refractive index gaps (such as air gaps) in the light path. Inaddition, the image uniformity of the image from the pinhole-lensassembly can be improved because the tilting angles of light raysentering the lens underneath the pinhole substrate are reduced with asmaller FOV due to the large refraction on the top of the pinholesubstrate 4621 e.

In the pinhole-lens assembly, the micro lens is placed underneath thepinhole 4643 and thus the optical aperture of the micro lens is smalldue to the small opening of the pinhole 4643. As such, the micro lensexhibits lower aberrations because light rays from the pinhole 4643collected by the micro lens generally are close to the axis of thecurved surfaces of the micro lens.

In implementing this pinhole-lens assembly, the center of the pinhole4643 is placed at or close to the center of the surface of the microlens 4621 e. In the example in FIG. 22, a half ball lens is shown as anexample and is engaged onto (e.g., being glued) a pinhole board toachieve this configuration. The flat surface of the half ball lens 4621e faces up to engage to the pinhole 643 and the center of the flatsurface of the half ball lens 4621 e is at or near the center of thepinhole 4643. Under this design, incident light, at both small or largeincident angles to the flat surface of the half ball lens 4621 e via thepinhole 4643, would have its light ray direction to coincide with aradial direction of the half ball lens 4621 e which is the optical axisof the lens in that direction. This configuration reduces opticalaberrations. For light beams 4663 and 4683 with different incidentangles at the top of the pinhole substrate 4621 f, their light paths aremodified after entering the pinhole substrate 4621 f to be close to therespective optical axes 4689 and 4691 of the half ball lens surface.Therefore, under this specific design, the image light spots 4681 and4693 formed by the light beams 4663 and 4683 exhibit low opticalaberrations although they have different incident angles.

The pinhole-lens assembly is subject to an aperture shading effect whichcauses the final image at the imaging plane (the optical sensor array4623 e) to appear brighter in the center and darker in the peripheralarea with a gradual change in brightness along the radial direction fromthe center towards the peripheral area. This effect degrades the imagecaptured at the optical sensor array 4623 e and can be reduced by usinga corrective optical filtering that modifies the spatial brightnessdistribution. For example, an optical filter with a spatial gradienttransmission profile can be inserted in the optical path of the lightreceived by the optical fingerprint sensor module, e.g., a locationbetween the OLED display panel and the optical sensor array. Thisgradient transmission filter is structured to exhibit a high opticalattenuation at or near a center of the pinhole and a decreasing opticalattenuation from the center of the pinhole radially outward to counter aspatial variation of an optical intensity distribution of light causedby the pinhole. FIG. 23 shows an example of an optical attenuationprofile for such a gradient transmission filter with a radial gradientattenuation that decreases from the center towards the edge.

In implementations, the gradient transmission filter may include one ormore coatings may be made on a surface of the light path to correct theimage brightness non-uniformity, e.g., the display bottom surface, themodule parts surface, or top surface of the optical sensor array. Inaddition to countering the spatial un-uniformity by the aperture shadingeffect, the filter may be further configured to correct other types ofbrightness non-uniformity and may also include features that can reduceother optical distortions and optical aberrations.

The above disclosed optical fingerprint sensor modules that uses apinhole-lens assembly for optical imaging onto an optical sensor arraymay also be used to construct optical fingerprint sensor modules locatedunder a top glass cover but is placed next to an LCD display screen thatis placed under the same top glass cover. Such optical fingerprintsensor modules can be placed next to an LCD display screen under thesame top glass cover to allow for a common and contiguous top surfaceabove the LCD display and the optical fingerprint sensor module andseveral examples are provided in the subsection VIII entitled “OpticalFingerprint Sensors on One Side of LCD Displays.”

IV. Invisible Under-LCD Optical Fingerprint Sensor Designs

According to some implementations, to make an under-display screenoptical fingerprint sensor module “invisible,” an optical fingerprintsensor module may be placed under the opaque border of the displayscreen. In many commercially produced LCD screens, an optically opaqueborder is present in the peripheral sides of the LCD screen, like aperipheral opaque border frame surrounding the central area of the LCDscreen. FIG. 24A illustrates a top view of the LCD display screen wherea peripheral opaque border frame 3220 is formed on the four sides of theLCD screen and block the LCD display pixels in the border regions on thefour sides of the LCD screen so that only the central LCD display area3210 exposed by the opening of the peripheral opaque border frame 3220is visible to a user during the display operation.

Accordingly, as illustrated in FIGS. 24A and 24B, the under-LCD opticalfingerprint sensor module can be placed under the LCD screen near orwithin the area covered by the peripheral opaque border frame 3220 sothat the LCD screen portion 3230 under the peripheral opaque borderframe 3220 above the optical fingerprint sensor module can be modifiedto provide one or more desired localized transmissive features orregions in the LCD screen stack for directing probe light carrying thefingerprint information to the optical fingerprint sensor module.

FIG. 24B shows a cross section view of the different layers under themarked circular region in FIG. 24A. The LCD screen portion 3230 withoptical transmissive features or regions for directing light to theoptical fingerprint sensor module is placed under the peripheral opaqueborder frame 3220 and thus is largely invisible to the user when viewingthe LCD-displayed images because the peripheral opaque border frame 3220is above and conceals the LCD screen portion 3230 from the view of theuser. Notably, different from some other examples of under-LCD opticalsensing designs, the center of the in-screen FPS sensing area 615 isspatially offset from the center of the LCD screen portion 3230 withoptical transmissive features or regions. Accordingly, the location ofthe underlying optical fingerprint sensor module is spatially shifted inposition off the in-screen FPS sensing area 615 because the underlyingoptical fingerprint sensor module is placed under the peripheral opaqueborder frame 3220 rather than being directly underneath the in-screenFPS sensing area 615.

This design of placing the LCD screen portion 3230 with opticaltransmissive features or regions either partially or entirely under theperipheral opaque border frame 3220 for directing probe light into theunder-LCD optical fingerprint sensor module is referred to as an“invisible” optical sensor design. This design can conceal both thelocalized transmissive features or regions in the LCD screen portion3230 of the LCD screen stack and the underlying optical fingerprintsensor module from being visible even when one or more extraillumination lights 4663 or 4661 are provided for illuminating the areaabove the in-screen FPS sensing area 615 where a user is to place afinger for optical fingerprint sensing and other optical sensingoperations.

Under this arrangement, the in-screen FPS sensing area 615 can no longerbe placed anywhere in the LCD screen but should be in the LCD screennear the location of the LCD screen portion 3230 having localizedtransmissive features or regions so that a portion of the light from thein-screen FPS sensing area 615 that reaches the LCD screen portion 3230under the peripheral opaque border frame 3220 can be directed throughthe LCD screen to reach the underlying optical fingerprint sensor modulefor optical sensing. In various implementations of this design, the LCDscreen portion 3230 having localized transmissive features or regions isdesigned to provide one or more light receiving paths (at leastpartially covered by the peripheral opaque border frame 3220) from thein-screen FPS sensing area 615 to the under-LCD optical fingerprintsensor module to improve the optical detection performance while theillumination of the in-screen FPS sensing area 615 can be implementedindependent of this special design. For example, the one or more extraillumination lights 4663 or 4661 can be placed at desired locations todirect illumination light to the in-screen FPS sensing area 615 withoutnecessarily going through the LCD screen portion 3230 having localizedtransmissive features or regions to reach the in-screen FPS sensing area615.

The above described design of placing the LCD screen portion 3230 havinglocalized transmissive features or regions to be at least partiallycovered by the peripheral opaque border frame 3220 can be applied tovarious configurations of the under-LCD optical fingerprint sensormodule for implementing the design in FIG. 24. For example, theunder-LCD optical fingerprint sensor module using a projector system forcollecting light from the in-screen FPS sensing area 615 in FIG. 4B, oran imaging system having a lens located below the LCD screen in anoptical path to the optical sensor array to collect the probe light fromthe in-screen FPS sensing area 615 and to project the collected probelight towards the optical sensor array in FIGS. 6B, 7, 8A, 9 and 10A, ora combination of a pinhole and a lens downstream from the pinhole toreceive the transmitted probe light from the pinhole and to focus thereceived probe light onto the optical sensor array for optical imagingas shown in FIGS. 17 through 23.

Referring back to FIGS. 24A-24B, the LCD screen portion 3230 havinglocalized transmissive features or regions that is partially or entirelycovered by the peripheral opaque border frame 3220 can be use variousdesigns to increase the optical transmission of the LCD screen portion3230 in addition to forming transmission holes or more transmissiveregions than the other parts of the LCD screen. FIGS. 25A-25C show onespecific example where a peripheral segment of the LCD screen ismodified to form the LCD screen portion 3230 by providing improvedoptical transmission in the modified LCD screen portion 3230 in whichthe underlying optical fingerprint sensor module 4620 as shown uses acombination of a pinhole and a lens shown in FIG. 17 to collect lightfor the optical sensor array as an example.

In this design example, as shown in FIGS. 25A and 25B, the under-LCDoptical fingerprint sensor module uses a pinhole 4643 and a micro lens4621 e together to form an optical system for collecting light from theFPS sensing area 615 on the top transparent layer 431 and for directingthe collected light onto the optical sensor array 4623 e in the opticalfingerprint sensor module 4620 to achieve a large field of view and ahigh imaging resolution at the same time. See FIGS. 17 through 23 andassociated description for more details. The spacer 4621 h may be alow-index material layer such as an air gap, and the protection layer4623 g may be a band pass filter. FIG. 25B shows that one or more undercover glass extra illumination light sources 4661 are provided toilluminate the finger to be detected and to generate probe light in theoptical path from the in-screen FPS sensing area 615 to the LCD screenportion 3230 to the optical fingerprint sensor module. Extra lightsources 4663 may be placed adjacent to or just above the opticalfingerprint sensor module 4620 to provide local illumination for fingerillumination. These light sources can also function as a breathing lightto indicate the operational state of the optical fingerprint sensormodule. As further explained in later sections, other light sources,such as environmental light sources, can also be used as probe lightsources for optical sensing.

FIGS. 25A-25C show that a peripheral segment of the LCD screen ismodified to form the LCD screen portion 3230 by peeling off a smallsegment of the LCD prism and diffusion films 4433 c and 4433 e and theother layers underneath the LCD prism and diffusion films 4433 c and4433 e. The peripheral segments of the LCD prism and diffusion films4433 c and 4433 e are removed to create a void for optical transmissionto form modified LCD prism and diffusion films 5433 c and 5433 e asshown in FIG. 25A. This void is used to insert an optical coupler 5617below the LCD layers 433 b and above the backlighting waveguide layer4433 f This optical coupler 5617 can be implemented as a wedged opticalcoupler with a tapered wedge section to extend into the space betweenthe peeled and modified LCD prism and diffusion films 5433 c and 4433 eand this tapered wedge section can reach into the LCD screen region 3100that is not covered by the peripheral opaque border frame 3220 as alight path for better collection of light from the in-screen FPS sensingarea 615. Accordingly, the optical coupler 5617 includes a top couplerpart that receives the light from the in-screen FPS sensing area 615 androutes the received light from the in-screen FPS sensing area 615 alonga slanted optical path to the region underneath the peripheral opaqueborder frame 3220 where the concealed optical fingerprint sensor module4620 is located to collect the received light from the in-screen FPSsensing area 615 for optical sensing. This feature of the opticalcoupler 5617 only impacts a small area of the LCD screen near theperipheral opaque border frame 3220 so that the impact to the displayquality is not pronounced. A thin spacer 5617 a is formed between thebottom surface of the LCD layers 4433 b and the top surface of theoptical coupler 5617 and may be, e.g., a soft optically clear ortransparent adhesive layer. As such, probe light from the in-screen FPSsensing area 615 passes through the thin spacer 5617 a and is collectedand directed by the optical coupler 5617 into the backlighting waveguidelayer 4433 f

In other implementations, the peripheral segment of the LCD screen isdetached or peeled off a small segment of the LCD prism and diffusionfilms 4433 c and 4433 e and the other layers underneath the LCD prismand diffusion films 4433 c and 4433 e to create a void for opticaltransmission to insert the optical fingerprint sensor module. Forexample, the backlighting module of the LCD display panel structureincludes a peripheral region within an area that is at least partiallycovered by the peripheral border and is detached from other parts of theLCD display panel structure to provide a location at which the opticalfingerprint sensor module is located under the peripheral opaque border.

Another modification to the peeled LCD peripheral segment is to remove asmall peripheral segment of the optical reflector film layer 4433 g as amodified optical reflector film layer 5433 g to expose the bottomsurface of the backlighting waveguide layer 4433 f for placing theoptical fingerprint sensor module 4620. The top surface of the opticalfingerprint sensor module 4620 in this example is the pinhole substrate4621 f which is placed against the bottom surface of the backlightingwaveguide layer 4433 f to receive the light directed by the opticalcoupler 5617. In this example, the optical path from the in-screen FPSsensing area 615 through the top transparent layer 431, through thetouch sensing layer 4433 a, the LCD layers 4433 b, the spacer 5617 a,the optical coupler 5617, and the backlighting waveguide layer 4433 finto the optical fingerprint sensor module 4620 contains no air gaps. Inother implementations, an air gap may be present in the above opticalpath.

FIG. 25B shows the general geometry of the LCD screen portion 3230 basedon the above modification of the peripheral segment of the LCD screenwhere two types of extra illumination light sources 4661 and 4663 areplaced below the top transparent layer 431 and above the backlightingwaveguide layer 4433 f.

FIG. 25C shows that, other than the peripheral segment of the LCD screenportion 3230, other parts of the LCD screen are not modified and areshown in their originally fabricated positions.

FIG. 26 shows an example of an implementation of the design in FIGS.25A-25C where the optical fingerprint sensor module 4620 is integratedunder the LCD assembly at a position close to the backlighting lightsources 434 at one end of the LCD screen structure. The light path forthe optical fingerprint sensor module 4620 is out of the LCD display'sworking zone (e.g., the actual LCD display area 3100) so that theoptical fingerprint sensor module 4620 is hidden under the LCD opaqueborder from the viewer and is substantially invisible. In this example,a LCD housing 4433 h below the modified LCD reflector film layer 5433 gis located above the optical fingerprint sensor module 4620 and has anoptical transparent window at the optical fingerprint sensor module 4620to allow the collected light to enter the optical fingerprint sensormodule 4620.

In summary, the above invisible optical fingerprint sensor module designfor an electronic device capable of detecting a fingerprint by opticalsensing (e.g., smartphones, tablets, and others) is based on a LCDscreen that provides touch sensing operations and includes a LCD displaypanel structure to display images and a peripheral opaque bordersurrounding a central area of the LCD display panel structure andcovering a narrow peripheral border of the LCD display panel structure.The LCD display panel structure includes a backlighting module toproduce backlight for illuminating the LCD layers to display images inthe central area of the

LCD display panel structure within the peripheral opaque border. One ormore optical sensing illumination probe light sources are provided toproduce probe light to illuminate a sensing area of the top transparentlayer for optical sensing. An under-LCD optical fingerprint sensormodule is located below the LCD screen and positioned underneath theperipheral opaque border to be spatially offset from central area of theLCD display panel structure to receive returned probe light that passesthrough the LCD screen for optical sensing. The LCD display panelstructure includes one or more extra transmission holes or regionswithin an area that is at least partially covered by the peripheralborder and is positioned above the optical fingerprint sensor module toallow probe light to pass through the LCD display panel structure toreach the optical fingerprint sensor module for optical sensing.

The one or more extra transmission holes or regions within the area thatis at least partially covered by the peripheral border may include, insome implementations, an optical coupler to transmit the light. In somedesigns, this optical coupler may be placed below the LCD layer andabove the LCD backlighting waveguide layer while the optical fingerprintsensor module is placed under the LCD backlighting waveguide layer at alocation where a portion of the LCD optical reflector film is removed orhas an opening or void. As shown by the example in FIG. 25A or 25B, oneimplementation of such an optical coupler is a tapered wedge thatdirects probe light, which is imprinted with fingerprint information, toa region above the optical fingerprint sensor module located under theLCD screen peripheral opaque border 3220 so that such probe light can becoupled into the optical fingerprint sensor module in a way generallyillustrated in FIG. 25A or 25B where the probe light enters the opticalsensor array of the optical fingerprint sensor module from the top ofthe optical fingerprint sensor module, either through the pinhole-lensassembly to the optical sensor array or directly onto the optical sensorarray.

V. Optical Fingerprint Sensor Module Including a Lens-Pinhole ImagingSystem with an Optical Axis Being Not Perpendicular to the DisplayScreen Surface

In some other implementations, the optical coupler 5671 illustrated inFIGS. 25A-25B may be omitted. Instead, the signal light may be directlycoupled into the optical fingerprint sensor module, thus simplifying thestructure for integrating the optical fingerprint sensor module to theLCD screen structure. FIGS. 27 and 28 illustrate an exemplaryimplementation of such a design for integrating an optical fingerprintsensor module without an optical coupler.

Referring to FIG. 27, an LCD display may include an LCD module 6002, atransparent cover 6003 (e.g., a glass cover) disposed over the LCDmodule 6002, and an LCD backlighting layers 6004 disposed under the LCDmodule 6002 for providing illumination for the LCD display. The LCDbacklighting layers 6004 may include layers such as LCD prism 5433 c anddiffusion films 5433 e as illustrated in FIG. 25A. The LCD display mayinclude an opaque zone 6006 on a periphery (i.e., the border) of the LCDdisplay.

In this implementation, a portion of the LCD backlighting layers 6004adjacent the opaque zone 6006 may be lifted slightly or be detached fromthe LCD module 6002 to create a space for placing an optical fingerprintsensor module 6000 under the opaque zone 6006 of the LCD module 6002.The LCD display may include a fingerprint sensing zone 6005. Signallight 6010 and 6012 from a finger placed adjacent the fingerprintsensing zone 6005 (e.g., scattered, reflected, or transmitted by thefinger) may be transmitted through the LCD module 6002, and may bereceived by the optical fingerprint sensor module 6000 at relativelylarge incidence angles. Thus, this design implementation may not requirethe optical coupler 5617 used in the implementation illustrated in FIGS.25A and 25B.

In some smartphone designs, this direct integration of the opticalfingerprint sensor module 6000 with the LCD display can operate tocapture fingerprint images by using signal light 6010 and 6012 atrelatively large incidence angles with respect to a directionperpendicular to the LCD screen surface in a range between 30 degreesand 90 degrees, e.g., 60 degrees to 85 degrees, or 70 degrees to 85degrees, according to various embodiments.

FIG. 28 shows an exemplary structure of an optical fingerprint sensormodule 6000 according some embodiments. The optical fingerprint sensormodule 6000 may include a FPC board 6090, and a photodiode array 6080disposed on and coupled to the FPC board 6090. The photodiode array 6080may be configured to convert signal light incident thereon intoelectrical signals. The FPC board 6090 may include electroniccircuitries for processing the electrical signals generated by thephotodiode array 6080 to produce images of fingerprint patterns carriedby the signal light. The photodiode array 6080 may be covered by aprotection layer 6082. In some embodiments, the protection layer 6082may be a bandpass filter or some other types of optical filter. In thisconfiguration, the optical fingerprint sensor module 6000 is flippedupside-down. That is, the FPC board 6090 and the photodiode array 6080are disposed adjacent the LCD module 6002 shown in FIG. 27.

The optical fingerprint sensor module 6000 may further include anoptically transparent spacer 6040 disposed above the protection layer6080. The spacer 6040 may have a relative low index of refraction. Insome embodiments, the spacer 6040 may be an air gap. The protectionlayer 6080 may have an index of refraction that is higher than that ofthe spacer 6040 according to some embodiments.

The optical fingerprint sensor module 6000 may further include a mirror6050 disposed above the spacer 6040 and extending beyond the spacer6040. As illustrated in FIG. 28, signal light 6010 scattered inside afinger may be reflected by the mirror 6050, and be received by thephotodiode array 6080. In addition, signal light 6018 reflected at theinterface between a top surface of the LCD display and a finger touchingthe fingerprint sensing zone 6005 of the LCD display may be reflected bythe mirror 6050, and be received by the photodiode array 6080. In someembodiments, the optical fingerprint sensor module 6000 may furtherinclude a light absorbing material 6062 disposed between the mirror 6052and the spacer 6040. The light absorbing material 6062 may be configuredto absorb stray light so as to reduce or eliminate background light.

In this implementation, the captured image at the photodiode array 6080may be spatially distorted due to the grazing incidence angles θ and θ′of the signal light 6010 and 6018 with respect to the normal 6084 of thesurface of the photodiode array 6080. Such spatial distortion may bemeasured. Based on the measured spatial distortion information, thegenerated detector signals from the photodiode array 6080 can beprocessed to correct the spatial distortion when reconstructing a finalimage.

In some embodiments, the spacer 6040 and the protection layer 6082 maybe configured to reduce the image distortion at the photodiode array6080. When light is refracted at interfaces between two media,nonlinearity in the directions of refracted rays may exist, which maycreate image distortions at the photodiode array 6080. Such distortionsmay be more pronounced when the incident angles are large. To reducesuch distortions, the ratio of the optical thickness of the spacer 6040and that of the protection layer 6080 may be selected in light of theoptical objective field.

As illustrated in FIG. 27, two additional illumination light sources6008 and 6009 may be provided to provide illumination to a finger intouch with or adjacent the fingerprint sensing area 6005 in order togenerate signal light 6010 and 6012 for optical sensing by the opticalfingerprint sensor module 6000. The illumination light source 6008 maybe placed at a position laterally displaced from the optical fingerprintsensor module 6000, to provide illumination light that may enter afinger to create scattered light 6010 and 6012 inside the finger. Thescatter light 6010 and 6012 may be transmitted through the LCD module6002 and be detected by the optical fingerprint sensor module 6000. Theillumination light sources may include light-emitting diodes (LEDs),vertical-cavity surface-emitting lasers (VCSELs), or laser diodes (LDs).Especially for illumination with VCSELs and LDs, the efficiency can behigher because the VCSEL or LD emitted light's divergence angle is smallso that the light is easier to be directed to fingertip. In addition,background light may also enter the finger and create scattered light,which may be transmitted through the LCD module 6002 to be combined withthe signal light 6010 and 6012 generated by the illumination of the oneor more extra illumination light sources 6008, and may be collected anddetected by the optical fingerprint sensor module 6000. One advantagefor using such signal light 6010 and 6012 may be that, as the signallight 6010 and 6012 is transmitted through or scattered by the innerissues of the finger, such signal light 6010 and 6012 tends to carryfingerprint information regardless of the finger skin conditions (e.g.,whether the skin is wet, dirty or dry) and may not be impacted by theinterface conditions between the finger and the top sensing surface.

Referring again to FIG. 27, the illumination light source 6009 may beplaced at a position inside or close to the optical fingerprint sensormodule 6000 to provide illumination to a finger that is firsttransmitted through the LCD module 6002 and the cover 6003 to interactwith a finger touching the top surface in the fingerprint sensing zone6005 and create back scattered light or reflected light (not shown inFIG. 27). Such signal light may be impacted by the skin conditions(e.g., the skin is wet, dirty or dry), and may also be impacted by theinterface conditions between the finger and the top sensing surface.

FIG. 29 shows an exemplary implementation of the design illustrated inFIGS. 27 and 61 where the optical fingerprint sensor module 6000 isintegrated under the LCD assembly at a position close to one or morebacklighting light sources 6480 at one end of the LCD screen structure.The LCD screen structure may include a top transparent layer 6003 and atouch sensing layer 6440. The LCD screen structure may further includeLCD layers 6450, an LCD prism layer 6460, and diffusion films 6470. Thecombination of the LCD layers 6450, the LCD prism layer 6460, and thediffusion films 6470 may be referred to as the LCD module 6002. The LCDscreen structure may further include the backlighting layers 6004, anLCD reflector film layer 6420, and an LCD housing 6410.

The optical fingerprint sensor module 6000 is hidden under the LCDopaque border 6006 from the viewer and is substantially invisible, asillustrated in FIG. 27. In this example, the LCD housing 6410 and theLCD reflector film layer 6420 are positioned below the opticalfingerprint sensor module 6000. The optical fingerprint sensor module6000 can be structured to be thin for better integration to the LCDstructure, e.g., around 1 millimeter or so in some implementations.

According to some embodiments, the spatial distortion caused by thegrazing incidence of the signal light 6010 and 6012 at the photodiodearray 6080 may be reduced by including a lens-pinhole assembly as shownin the examples illustrated in FIGS. 30-32.

As discussed above with references to FIGS. 17 through 26, a compactoptical imaging system for optical fingerprint sensing may be providedby combining a lens and a pinhole as a lens-pinhole imaging system. Thelens is used to form a lens-based imaging system to achieve a highspatial image resolution, as well as a reduced size in the capturedimage at the photodiode array 6080 to reduce the size of the photodiodearray. The pinhole may be placed in front of the lens to produce a largefield of view (FOV) in optical imaging by effectuating a pinhole camera.In the examples illustrated in FIGS. 17 through 26, the lens-pinholeimaging system is implemented to have the optical axis of thelens-pinhole imaging system to be approximately perpendicular to the LCDscreen surface and the photodiode array surface (i.e., the optical axisof the lens-pinhole imaging system is approximately parallel to thenormal of the photodiode array surface). For example, in the invisibleunder-LCD optical fingerprint sensor module illustrated in FIGS. 25A and25B, which includes an optical wedge coupler 5617, the optical axis ofthe lens-pinhole imaging system is approximately perpendicular to theLCD screen surface and the photodiode array surface. FIGS. 30-32illustrate exemplary implementations of a different approach to thelens-pinhole imaging system in which the optical axis of thelens-pinhole imaging system is nearly parallel to the LCD screen surfaceand the photodiode array surface.

FIG. 30 illustrates an optical fingerprint sensor module 6300 accordingto some embodiments. The optical fingerprint sensor module 6300 mayinclude a FPC board 6090, and a photodiode array 6080 disposed on andcoupled to the FPC board 6090. The photodiode array 6080 may be coveredby a protection layer 6082. In some embodiments, the protection layer6082 may be a bandpass filter or some other types of optical filter.

The optical fingerprint sensor module 6300 further includes alens-pinhole assembly. The lens-pinhole assembly includes a pinholesubstrate 6032, a pinhole 6030 formed on the pinhole substrate 6032, anda micro lens 6020 disposed behind the pinhole 6030. The lens-pinholeassembly is positioned such that the optical axis 6034 of the lens 6020is off-normal with respect to the surface of the photodiode array 6080.The optical axis of a lens may be defined as a line passing through thecenter of curvature of the lens and parallel to the axis of symmetry.The angle β between the optical axis 6034 of the lens 6020 and thenormal of the surface of the photodiode array 7080 may be optimized toincrease the effective aperture. In some embodiments, the angle β mayrange from about 45 degrees to about 135 degrees, or from about 80degrees to about 95 degrees. In some embodiments, the angle β may beabout 90 degrees. In such cases, the optical axis 6034 of thelens-pinhole assembly may be nearly parallel to the surface of thephotodiode array 6080 (i.e., the angle β is nearly 90 degrees). In someembodiments, optical bandpass filter coatings may be formed on thepinhole substrate 6032 or on the surfaces of other components.

The optical fingerprint sensor module 6300 may be positioned under theopaque border 6006 of the LCD module 6002. An optically transparentspacer 6040 may be positioned between the LCD module 6002 and theprotection layer 6082. The spacer 6040 may have a relatively low indexof refraction. In some embodiments, the spacer 6040 comprises an airgap. The protection layer 6082 may have an index of refraction that ishigher than that of the spacer 6040.

The spacer 6040 and the protection layer 6082 may be configured toreduce the imaging distortions at the surface of the photodiode array6080. When light is refracted at interfaces between two media,nonlinearity in the directions of refracted rays may exist, which maycreate image distortions at the photodiode array 6080. Such distortionsmay be more pronounced when the incidence angles are large. To reducesuch distortions, the ratio of the optical thickness of the spacer 6040and that of the protection layer 6080 can be selected in light of theoptical structure of the pinhole-lens assembly and the optical objectivefield of the pinhole-lens assembly.

The lens-pinhole assembly is positioned on the left hand side of theoptical fingerprint sensor module 6300 to collect incident signal light6010 and 6012 at large incidence angles due to the placement of theoptical fingerprint sensor module 6300 under the opaque border 6006 ofthe LCD module 6002. The pinhole 6030 first receives the incident signallight 6010 and 6012, and the micro lens 6020 then images the signallight that passes the pinhole 6030 onto the surface of the photodiodearray 6080. The spatial distortions due to the large incidence angles ofthe signal light 6010 and 6012 may be reduced by using the lens-pinholeimaging system. For example, the micro lens 6020 may be shaped to reduceor counter the distorted spatial distribution of the incident signallight 6010 and 6012. The residual spatial distortion may be measured forthe given the lens-pinhole assembly, so that signal processing may beapplied to reduce or remove the residual distortion in reconstructing animage of a fingerprint pattern.

FIG. 31 illustrates an optical fingerprint sensor module 6400 accordingto some other embodiments. Similar to the optical fingerprint sensormodule 6300, the optical fingerprint sensor module 6400 may include aFPC board 6090, a photodiode array 6080 disposed on and coupled to theFPC board 6090, a protection layer 6082 disposed over the photodiodearray 6080, and a lens-pinhole assembly that includes a pinhole 6030 anda micro lens 6030. In some embodiments, the protection layer 6082 may bea bandpass filter or other some other types of optical filter.

In this configuration, the optical fingerprint sensor module 6400 isflipped upside-down as compared to the optical fingerprint sensor module6300 illustrated in FIG. 30. That is, the FPC board 6090 and thephotodiode array 6080 are disposed adjacent the LCD module 6002 (theyare shown as directly above the LCD module 6002 because the figure isflipped upside-down).

Additionally, the optical fingerprint sensor module 6400 may include amirror 6050 disposed above the spacer 6040 and the lens-pinholeassembly. The mirror 6050 extends beyond the front of the pinholesubstrate 6032 to form a ledge. The signal light 6010 and 6012transmitted through the LCD module 6002 may be incident on the ledgeportion of the mirror 6050 and be reflected by the mirror 6050. Thereflected signal light may in turn be imaged by the lens-pinholeassembly onto the surface of the photodiode array 6080. Because thelight paths of the signal light 6010 and 6012 are folded by the mirror6050, the optical fingerprint sensor module 6400 may be made relativelythin, as compared to the optical fingerprint sensor module 6300illustrated in FIG. 30.

FIG. 32 illustrates an optical fingerprint sensor module 6500 accordingto some other embodiments. The optical fingerprint sensor module 6500 issimilar to the optical fingerprint sensor module 6400, illustrated inFIG. 31, in that it is also flipped upside-down. Here, instead of usinga mirror 6050, the optical fingerprint sensor module 6500 may include amicro prism 6070 disposed in front of the lens-pinhole assembly. Theincident signal light 6010 and 6012 may be transmitted through a firstsurface 6072 of the micro prism 6070 and be reflected at a secondsurface 6074 of the micro prism 6070. The reflected signal light maythen be transmitted through a third surface 6076 of the micro prism6070, to be received by the lens-pinhole assembly. Because the lightpaths of the signal light 6010 and 6012 are folded by the micro prism6070, the optical fingerprint sensor module 6500 may be made relativelythin, similar to the optical fingerprint sensor module 6400 illustratedin FIG. 31.

In some embodiments, the micro prism 6070 may comprise a materialselected to have an index of refraction such that the signal light 6010and 6012 incident on the second surface 6074 of the micro prism 6070 mayundergo total internal reflection (TIR). In some other embodiments, thesecond surface 6074 of the micro prism may be coated with a highreflective material, such as a metal, so that the second surface 6074may function as a mirror. In some embodiments, the orientation of thefirst surface 6072 of the micro prism may be configured such that thesignal light 6010 and 6012 is incident on the first surface 6072 closeto normal incidence. Additionally, the first surface 6072 may be coatedwith an anti-reflection coating so as to reduce attenuation of theintensity of the signal light 6010 and 6012 by reflection. Similarly,the third surface 6076 may also be coated with an anti-reflectioncoating to reduce reflection.

FIG. 33 shows different light signals that may be present in a devicethat implements the invisible under-LCD optical sensing design disclosedin connection with the examples illustrated in FIGS. 27-32. In theillustrated example in FIG. 33, one or more extra light sources 6008 maybe placed at one side of the LCD module near the fingerprint sensingarea 6005 (as shown in FIG. 27) on the top of the top transparent layer6003, to produce illumination light for optical sensing. For example, anillumination light beam 6620 may pass through the top transparent layer6003 to illuminate a touching finger 6610 at the fingerprint sensingarea 6005 (as illustrated in FIG. 27). A portion of the light from theillumination light beam 6620 may enter the finger 6610 and be scatteredby the finger tissues. A portion of the scattered light (e.g., 6640) maybe transmitted through the finger 6610 and incident on the toptransparent layer 6003 in the fingerprint sensing area 6005, and may betransmitted through the top transparent layer 6003 to be collected bythe optical fingerprint sensor module 6000. As described above, theportion of the scattered light 6640 that comes out of the finger to betransmitted through the top transparent layer 6003 may carry thefingerprint information, and thus may be detected to extract the userfingerprint information.

FIG. 33 further shows one or more illumination light sources 6009 thatare located adjacent to the optical fingerprint sensor module 6000 andare under the LCD module. The light from such an illumination lightsource 6009 may be directed to the top transparent layer 6003 by passingthrough the LCD module. Referring back to FIGS. 5A-5C, the light fromthe illumination light source 6009 at the fingerprint sensing area 6005may encounter the finger ridges 61 (e.g., light rays 80, 201) andvalleys 63 (e.g., light rays 82, 211 and 212) to cause reflections 181,205 and 206 from the ridges 61 and reflections 185, 213 and 214 from thevalleys 63 from the top surface of the top transparent layer 6003 incontact with the finger 6610. The reflection rays from the differentlocations have different signal amplitudes and thus are imprinted withthe fingerprint pattern as a 2-D fingerprint pattern. In addition, partof each of incident light rays from below the top transparent layer 6003may enter the finger, e.g., light rays 183 from the light rays 80, lightrays 189 from the light rays 82, light rays 203 from light rays 201 andlight rays 204 from light rays 202, and may be scattered by internalfinger tissues to produce scattered light 191 towards the toptransparent layer 6003 which may be received by the optical fingerprintsensor module 6000. Similar to the portion 6640 of the scattered lightthat comes out of the finger 6610 to be transmitted through the toptransparent layer 6003 in FIG. 33, the scattered light 191 caused by thescattering in FIGS. 5A and 5B due to illumination light from theillumination light sources 6009 carries the fingerprint information andthus can be detected to extract the user fingerprint information.

In some applications, the illumination of the one or more extraillumination light sources 6008 may be used without having theillumination of the one or more extra illumination light sources 6009;in other applications, the illumination of the one or more extraillumination light sources 6009 may be used without having theillumination of the one or more extra illumination light sources 6008.In yet other implementations, both extra illumination light sources 6008and 6009 may be used.

FIG. 33 illustrate additional illumination light sources forilluminating a finger 6610. The incident light 6630 may be present inthe environment such as natural sky light, sunlight or other lightsources in the environment to illuminate the touching finger 6610. Theincident light 6630 may be scattered by the internal tissue within thefinger 6610 to result in scattered light as signal light 6650. Thesignal light 6650 may exit the finger 6610 to carry both the fingerprintpattern information and the additional topographical information fromthe internal tissue structures below the finger skin. The additionaltopographical information from the internal tissue structures below thefinger skin may be valuable information for fingerprint sensing, and isthree-dimensional since the internal tissue structures vary with boththe lateral position under the skin and the depth from the skin surface(topographical information). Such additional topographical informationfrom the internal tissue structures of a finger can be used, forexample, to improve the imaging reliability under varying finger surfaceor finger-glass conditions, and to distinguish a natural finger from anartificial object made with similar or identical external fingerprintpattern as the natural finger.

FIGS. 34A and 34B illustrate an implementation of an optical fingerprintsensor module 7000 according to some embodiments. The opticalfingerprint sensor module 7000 may include a FPC board 7090, aphotodiode array 7080 disposed on and coupled to the FPC board 7090. Thephotodiode array 7080 may be covered by a protection layer 7082. In someembodiments, the protection layer 7082 may be a bandpass filter or othersome other types of optical filter.

The optical fingerprint sensor module 7000 further includes alens-pinhole assembly. The lens-pinhole assembly includes a pinholesubstrate 7032, a pinhole 7030 formed on the pinhole substrate 7032, anda micro lens 7020 in front of the pinhole 7030. Similar to the opticalfingerprint sensor module 6300, 6400, and 6500 illustrated in FIGS.30-32, the lens-pinhole assembly is positioned such that the opticalaxis 7034 of the lens 7020 is off-normal with respect to the surface ofthe photodiode array 7080. The angle between the optical axis 7034 ofthe lens 7020 and the normal of the surface of the photodiode array 7080may be optimized to increase the effective aperture. In someembodiments, the angle between the optical axis 7034 of the lens 7020and the normal of the surface of the photodiode array 7080 may rangefrom about 45 degrees to about 135 degrees, or from about 80 degrees toabout 95 degrees. In some embodiments, the optical axis 7034 of the lens7020 may be nearly parallel to the surface of the photodiode array 7080(i.e., angle between the optical axis of the lens and the normal of thesurface of the photodiode array 7080 is about 90 degrees). In someembodiments, an optical bandpass filter coating may be formed on thepinhole substrate 7032 or on the surfaces of other components.

The optical fingerprint sensor module 7000 may further include a supportplate 7062 positioned above the photodiode array 7080 and thelens-pinhole assembly. An optically transparent spacer 7040 separatesthe support plate 7062 and the protection layer 7080 covering the top ofthe photodiode array 7080. The spacer 7040 may have a relatively lowindex of refraction. In some embodiments, the spacer 7040 may be an airgap. The protection layer 7082 may have an index of refraction that ishigher than that of the spacer 7040.

The spacer 7040 and the protection layer 7082 may be configured toreduce the image distortions at the surface of the photodiode array7080. When light is refracted at interfaces between two media,nonlinearity in the directions of refracted rays may exist, which maycreate image distortions at the surface of the photodiode array 7080.Such distortions may be more pronounced when the incidence angles arelarge. To reduce such distortions, the ratio of the optical thickness ofthe spacer 7040 and that of the protection layer 7080 may be selected inlight of the optical structure of the pinhole-lens assembly and theoptical objective field of the pinhole-lens assembly.

In some embodiments, an absorbing coating 7060 may be applied to aportion of the surface of the support plate 7062 that is locateddirectly above the photodiode array 7080 and behind the lens-pinholeassembly. The absorbing coating 7060 may be configured to absorb straylight so as to reduce or eliminate background light.

Similar to the optical fingerprint sensor module 6400 illustrated inFIG. 31 and the optical fingerprint sensor module 6500 illustrated inFIG. 32, the optical fingerprint sensor module 7000 is flippedupside-down as compared to the optical fingerprint sensor module 6300illustrated in FIG. 30. That is, the photodiode array 7080 is disposedadjacent the display cover 7002 of an LCD module, as illustrated in FIG.34B.

Referring to FIGS. 34A and 34B, a portion of the support plate 7060 mayextend beyond the front of the lens-pinhole assembly to form a ledge. Amirror 7050 may be attached to the ledge portion of the support plate7062, so that signal light 7010 and 7012 from a finger may betransmitted by the display cover 7002 and be reflected by the mirror7050 toward the lens-pinhole assembly. The reflected signal light may berefracted by the micro lens 7020 and pass through the pinhole 7030, andthe refracted signal light 7014 and 7016 may be incident on the surfaceof photodiode array 7080. Similar to the optical fingerprint sensormodule 6400 illustrated in FIG. 31 and the optical fingerprint sensormodule 6500 illustrated in FIG. 32, the overall thickness of the opticalfingerprint sensor module 7000 may be made relatively thin by foldingthe light path of the signal light 7010 and 7012 using the mirror 7050.

In this implementation, the pinhole 7030 is positioned off of theoptical axis 7034 of the micro lens 7020. For example, the pinhole 7030is positioned near the upper edge of the pinhole substrate 7032 adjacentthe support plate 7062. As such, the refracted signal light 7014 and7016 may be incident on the surface of photodiode array 7080 at anglesof incidence θ and θ′ that are smaller than those if the pinhole 7030 ispositioned lower to be aligned with the optical axis 7034 of the microlens 7020. For example, comparing to the optical fingerprint sensormodule 6400 illustrated in FIG. 31 where the pinhole 6030 is alignedwith the optical axis 6034 of the micro lens 6020, the angles ofincidence α and α′ of the refracted signal light 6014 and 6016 may berelatively large (i.e., at higher grazing angles); whereas in theoptical fingerprint sensor module 7000 illustrated in FIGS. 34A-34B,because the location of the pinhole 7030 is farther away from thesurface of the photodiode array 7080, the angles of incidence θ and θ′of the refracted signal light 7014 and 7016 may be smaller than theangles of incidence α and α′ of the refracted signal light 6014 and 6016illustrated in FIG. 31. In other words, the refracted light signals 7014and 7016 are incident on the surface of the photodiode array 7080 closerto normal incidence.

The configuration of the optical fingerprint sensor module 7000illustrated in FIGS. 34A and 34B may afford several advantages. Forexample, because photodiodes may generally have higher detectionefficiencies at smaller angles of incidence as compared to larger anglesof incidence, the detection efficiency of the photodiode array 7080 maybe higher in this configuration as compared to the configuration of theoptical fingerprint sensor module 6400 illustrated in FIG. 31. Inaddition, as illustrated in FIG. 34B, the fan of light bounded by themarginal rays of the signal light 7010 and 7012 that may pass throughthe lens-pinhole assembly to be detected by the photodiode array 7080may be “flatter.” That is, the angles ϕ and ϕ′ between the signal light7010 and 7012 and the surface of the LCD module 7002 may be smaller thanthose if the pinhole 7030 is positioned lower to be aligned with theoptical axis of the micro lens 7020. As a result, the lateral field ofview (FOV) 7009 (i.e., the effective sensing area of the fingerprintsensing zone 7005 subtended by the two marginal signal light rays 7010and 7012) may be larger.

In some embodiments, the optical fingerprint sensor module 7000 mayfurther include a light extinction region 7070 on the ledge portion ofthe support plate 7062 adjacent the lens-pinhole assembly. The lightextinction region 7070 may be configured to attenuate the intensity ofthe signal light 7010 on the near side of the FOV 7009 (corresponding tothe marginal signal light 7010 represented by the solid arrow). Sincethe signal light 7010 may be brighter than the signal light 7012 on thefar side of the FOV 7009 (e.g., because the target is closer), the lightextinction region 7070 may balance the image light intensity across thesurface of the photodiode array 7080. Therefore, the light extinctionregion 7070 may serve as an aperture filter.

In some embodiments, the focal length of the micro lens 7020 may beconfigured such that the signal light rays 7012 on the far side of theFOV 7009 may be sharply focused on the surface of the photodiode array7080. Since the effective aperture size of the pinhole 7030 may besmaller for the signal light ray 7010 on the near side of the FOV 70009due to the effect of the light extinction region 7070, the signal lightrays 7010 on the near side of the FOV 7009 may also be sharply focusedon the photodiode array 7080. Thus, in this manner, the image on thesurface of the photodiode array 7080 may have a relatively uniformspatial resolution across the surface of the photodiode array 7080.

The implementation illustrated in FIGS. 34A and 34B mostly rely on theeffect of the pinhole 7030 for imaging. Therefore, distortion may beminimized, even though the overall thickness of the under-LCD opticalfingerprint sensor module 7000 is made relatively thin. The micro lens7020 may serve to improve the image contrast (e.g., by improving theimage spatial resolution).

FIGS. 35A and 35B illustrate an implementation of an optical fingerprintsensor module 7100 according to some embodiments. Similar to the opticalfingerprint sensor module 7000, the optical fingerprint sensor module7100 may include a FPC board 7090, a photodiode array 7080 disposed onand coupled to the FPC board 7090. The photodiode array 7080 may becovered by a protection layer 7082. In some embodiments, the protectionlayer 7082 may be a bandpass filter or other some other types of opticalfilter.

The optical fingerprint sensor module 7100 further includes alens-pinhole assembly. The lens-pinhole assembly includes a pinholesubstrate 7032, a pinhole 7030 formed on the pinhole substrate 7032, anda micro lens 7020 in front of the pinhole 7030. The lens-pinholeassembly is positioned such that the optical axis 7034 of the lens 7020is off-normal with respect to the surface of the photodiode array 7080.The angle between the optical axis 7034 of the lens 7020 and the normalof the surface of the photodiode array 7080 may be optimized to increasethe effective aperture. In some embodiments, the angle between theoptical axis 7034 of the lens 7020 and the normal of the surface of thephotodiode array 7080 may range from about 45 degrees to about 135degrees, or from about 80 degrees to about 95 degrees. In someembodiments, the optical axis 7034 of the lens 7020 may be nearlyparallel to the surface of the photodiode array 7080 (i.e., anglebetween the optical axis of the lens and the normal of the surface ofthe photodiode array 7080 is about 90 degrees). In some embodiments,optical bandpass filter coatings may be formed on the pinhole substrate7032 or on the surfaces of other components.

Similar to the optical fingerprint sensor module 7000 illustrated inFIG. 34A, the optical fingerprint sensor module 7100 is flippedupside-down as compared to the optical fingerprint sensor module 6300illustrated in FIG. 30. That is, the photodiode array 7080 is disposedadjacent the display cover 7002 of an LCD module, as illustrated in FIG.35B.

Referring to FIGS. 35A and 35B, the optical fingerprint sensor module7100 may further include a support plate 7162 positioned above thephotodiode array 7080. Unlike the support plate 7062 in the opticalfingerprint sensor module 7000, the support plate 7162 does not extendbeyond the lens-pinhole assembly. The optical fingerprint sensor module7100 may further include a mirror 7150 formed on a portion of a surfaceof the support plate 7162 just behind the lens-pinhole assembly. Thepinhole 7030 and the micro lens 7020 are positioned near the upper edgeof the pinhole substrate 7032, just below the support plate 7162. Inaddition, the pinhole 7030 is positioned off of the optical axis 7034 ofthe micro lens 7020. Signal light 7010 and 7012 from a finger may betransmitted through the LCD module 7002 and be refracted by the microlens 7020 and transmitted through the pinhole 7030. The refracted signallight 7014 and 7016 may be incident on the mirror 7150 and be reflectedby the mirror 7150 toward the surface of the photodiode array 7080.

In this implementation, the overall thickness of the optical fingerprintsensor module 7100 may be made relatively thin by folding the light pathof the signal light 7010 and 7012 using the mirror 7150. Because thepinhole 7030 and the micro lens 7020 are positioned higher and closer tothe mirror 7150, the refracted signal light 7014 and 7016 may beincident on the photodiode array 7080 at angles of incidence θ and θ′that are smaller than those if the pinhole 7030 is positioned lower. Inother words, the refracted light signals 7014 and 7016 are incident onthe photodiode array 7080 closer to normal incidence. Becausephotodiodes may generally have higher detection efficiencies at smallerangles of incidence as compared to larger angles of incidence, thedetection efficiency of the photodiode array 7080 may be higher.

In addition, as illustrated in FIG. 35B, the fan of light bounded by themarginal rays of the signal light 7010 and 7012 that may pass throughthe lens-pinhole assembly to be detected by the photodiode array 7080may be “flatter.” That is, the angles ϕ and ϕ′ between the signal light7010 and 7012 and the surface of the LCD module 7002 may be smaller thanthose if the pinhole 7030 is positioned lower to be aligned with theoptical axis of the micro lens 7020. As a result, the lateral field ofview (FOV) 7109 (i.e., the effective sensing area of the fingerprintsensing zone 7005 subtended by the two marginal signal light rays 7010and 7012) may be larger.

In some embodiments, the optical fingerprint sensor module 7100 mayfurther include a light extinction region 7170 positioned behind thelens-pinhole assembly and in front of the mirror 7150. The lightextinction region 7170 may be configured to attenuate the intensity ofthe signal light 7010 on the near side of the FOV 7109 (corresponding tothe marginal signal light 7010 represented by the solid arrow). Sincethe signal light 7010 may be brighter than the signal light 7012 on thefar side of the FOV 7109 (corresponding to the marginal signal light7012 represented by the dashed arrow), the light extinction region 7070may balance the image light intensity across the photodiode array 7080.Therefore, the light extinction region 7070 may serve as an aperturefilter.

In some embodiments, an absorbing coating 7160 may be applied to aportion of the support plate 7062 next to the mirror 7150. The absorbingcoating 7160 may be configured to absorb stray light so as to reduce oreliminate background light.

In some embodiments, the focal length of the micro lens 7020 may beconfigured such that the signal light rays 7012 on the far side of theFOV 7109 may be sharply focused on the photodiode array 7080. Since theeffective aperture size of the pinhole 7030 may be smaller for thesignal light ray 7010 on the near side of the FOV 7109 due to the effectof the light extinction region 7070, the signal light rays 7010 on thenear side of the FOV 7109 may also be sharply focused on the photodiodearray 7080. Thus, in this manner, the image on the surface of thephotodiode array 7080 may have relatively uniform spatial resolutionacross the surface of the photodiode array 7080.

FIGS. 36A and 36B illustrate an implementation of an optical fingerprintsensor module 7200 according to some embodiments. The opticalfingerprint sensor module 7200 may include a FPC board 7090, aphotodiode array 7080 disposed on and coupled to the FPC board 7090. Thephotodiode array 7080 may be covered by a protection layer 7082. In someembodiments, the protection layer 7082 may be a bandpass filter or othersome other types of optical filter.

The optical fingerprint sensor module 7200 may also include alens-pinhole assembly. The lens-pinhole assembly includes a pinholesubstrate 7032, a pinhole 7030 formed on the pinhole substrate 7032, anda micro lens 7020 disposed in front of the pinhole 7030. Thelens-pinhole assembly may be positioned such that the optical axis 7034of the lens 7020 is off-normal with respect to the surface of thephotodiode array 7080. In some embodiments, the angle between theoptical axis 7034 of the lens 7020 and the normal of the surface of thephotodiode array 7080 may range from about 45 degrees to about 135degrees, or from about 80 degrees to about 95 degrees. In someembodiments, the angle between the optical axis 7034 of the lens 7020and the normal of the surface of the photodiode array 7080 may be about90 degrees. In some embodiments, optical bandpass filter coatings may beformed on the pinhole substrate 7032 or on the surfaces of othercomponents.

The optical fingerprint sensor module 7200 may further include a mirrorholder 7254 positioned under the FPC board 7090 and extending to infront of the lens-pinhole assembly, and a first mirror 7252 formed onthe portion of the mirror holder 7254 that extends in front of thelens-pinhole assembly. In some embodiments, the first mirror 7252 may beformed by applying a reflective coating to a portion of the surface ofthe mirror holder 7254. In some other embodiments, the mirror holder7254 may be disposed between the FPC board 7090 and the photodiode array7080 (not shown in FIG. 36A).

Referring to FIG. 36B, the optical fingerprint sensor module 7200 may bedisposed between a screen 7202 and a back light module 7204 of an LCDdisplay module. The first mirror 7252 is adjacent the back light module7204. The optical fingerprint sensor module 7200 may be positioned underan opaque border 7206 of the screen 7202 so that it is invisible. Signallight 7011 and 7013 scattered by a finger (not shown in FIG. 36B) placedabove and adjacent the screen 7202 may be transmitted by the screen 7202and be incident on the first mirror 7252. The first mirror 7252 mayreflect the signal light 7011 and 7013 toward the micro lens 7020 andthe pinhole 7030. Signal light 7011 and 7013 may be refracted by themicro lens 7020 and be transmitted through the pinhole 7030.

Referring to FIG. 36A, the optical fingerprint sensor module 7200 mayfurther include a support plate 7162 positioned above the photodiodearray 7080, and a second mirror 7150 formed on a portion of a surface ofthe support plate 7162 that faces the photodiode array 7080. In someembodiments, the second mirror 7150 may be formed by applying areflective coating to a portion of the surface of the support plate7162. The second mirror 7150 is positioned just behind the lens-pinholeassembly. Signal light 7011 and 7013 transmitted though the pinhole 7030may be reflected by the second mirror 7150 toward the surface of thephotodiode array 7080.

In some embodiments, the pinhole 7030 and the micro lens 7020 arepositioned near the upper edge of the pinhole substrate 7032, just belowthe support plate 7162. In addition, the pinhole 7030 may be positionedoff of the optical axis 7034 of the micro lens 7020. Because the pinhole7030 and the micro lens 7020 are positioned closer to the second mirror7150 and farther from the surface of the photodiode array 7080, thesignal light 7014 and 7016 transmitted through the pinhole 7030 may beincident on the photodiode array 7080 closer to normal incidence.Because photodiodes generally have higher detection efficiencies atsmaller angles of incidence as compared to larger angles of incidence,the detection efficiency of the photodiode array 7080 may be higher.

The optical fingerprint sensor module 7200 may further include a lightextinction region 7170 positioned behind the pinhole 7030 and in frontof the second mirror 7150. The light extinction region 7170 may beconfigured to attenuate the intensity of signal light 7011 on the nearside of the FOV 7209 (as illustrated in FIG. 36B, corresponding to themarginal signal light 7011 represented by the solid arrow). Since signallight 7011 on the near side of the FOV 7209 may be brighter than signallight 7013 on the far side of the FOV 7209 (corresponding to themarginal signal light 7013 represented by the dashed arrow), the lightextinction region 7070 may balance the image light intensity across thephotodiode array 7080. Therefore, the light extinction region 7070 mayserve as an aperture filter.

The optical fingerprint sensor module 7200 may further include anabsorbing coating 7160 applied to another portion of the surface of thesupport plate 7062 abutting the second mirror 7150. The absorbingcoating 7160 may be configured to absorb stray light so as to reduce oreliminate background light.

Referring to FIG. 36B, by having two mirrors, namely the first mirror7252 and the second mirror 7150, signal light 7011 and 7013 are foldedtwice, once by the first mirror 7252 and again by the second mirror7150. Therefore, the lateral FOV 7209 of the optical fingerprint sensormodule 7200 may be expanded as compared to the FOV 7109 of the opticalfingerprint sensor module 7100 illustrated in FIG. 35B, although theangular FOV is the same. By folding the optical path multiple times, theoptical fingerprint sensor module 7200 may be made thinner, and thusmore suitable to be installed between the screen 7202 and the back lightmodule 7204 of the LCD display module.

While this disclosure contains many specifics, these should not beconstrued as limitations on the scope of any invention or of what may beclaimed, but rather as descriptions of features that may be specific toparticular embodiments of particular inventions. Certain features thatare described in this patent document in the context of separateembodiments can also be implemented in combination in a singleembodiment. Conversely, various features that are described in thecontext of a single embodiment can also be implemented in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. Moreover, the separation of various system components in theembodiments described in this patent document should not be understoodas requiring such separation in all embodiments.

Only a few implementations and examples are described and otherimplementations, enhancements and variations can be made based on whatis described and illustrated in this patent document.

A recitation of “a”, “an” or “the” is intended to mean “one or more”unless specifically indicated to the contrary.

Ranges may be expressed herein as from “about” one specified value,and/or to “about” another specified value. The term “about” is usedherein to mean approximately, in the region of, roughly, or around. Whenthe term “about” is used in conjunction with a numerical range, itmodifies that range by extending the boundaries above and below thenumerical values set forth. In general, the term “about” is used hereinto modify a numerical value above and below the stated value by avariance of 10%. When such a range is expressed, another embodimentincludes from the one specific value and/or to the other specifiedvalue. Similarly, when values are expressed as approximations, by use ofthe antecedent “about,” it will be understood that the specified valueforms another embodiment. It will be further understood that theendpoints of each of the ranges are included with the range.

All patents, patent applications, publications, and descriptionsmentioned here are incorporated by reference in their entirety for allpurposes. None is admitted to be prior art.

What is claimed is:
 1. An optical fingerprint sensor module comprising:a light source configured to provide illumination light directed towarda finger, a portion of the illumination light being scattered orreflected off of the finger, thereby generating signal light; aphotodiode array having a surface; an optically transparent spacerdisposed over the surface of the photodiode array; a first mirrorconfigured to reflect the signal light; a lens configured to receive andrefract the signal light reflected by the first mirror, the lens havingan optical axis that forms an angle with respect to a normal of thesurface of the photodiode array that is between 45 degrees and 135degrees; a member defining a pinhole disposed behind the lens, thepinhole configured to transmit the signal light refracted by the lens; asecond mirror disposed behind the pinhole and above the opticallytransparent spacer, the second mirror configured to reflect the signallight transmitted through the pinhole toward the surface of thephotodiode array; and electronic circuitries electrically coupled to thephotodiode array, wherein the photodiode array is configured to convertthe signal light incident thereon into electrical signals, and theelectronic circuitries are configured to process the electrical signalsto produce an image of a fingerprint pattern of the finger.
 2. Theoptical fingerprint sensor module of claim 1, wherein the pinhole ispositioned off of the optical axis of the lens.
 3. The opticalfingerprint sensor module of claim 1, wherein the angle between theoptical axis of the lens and the normal of the surface of the photodiodearray is between 80 degrees and 95 degrees.
 4. The optical fingerprintsensor module of claim 1, wherein the angle between the optical axis ofthe lens and the normal of the surface of the photodiode array is about90 degrees.
 5. The optical fingerprint sensor module of claim 4, whereina distance between the optical axis of the lens and the surface of thephotodiode array is greater than a distance between the optical axis ofthe lens and the second mirror.
 6. The optical fingerprint sensor moduleof claim 1, further comprising a light extinction region disposedadjacent the pinhole and abutting the second mirror, the lightextinction region configured to attenuate a portion of the signal lighttransmitted through the pinhole and incident on the light extinctionregion.
 7. The optical fingerprint sensor module of claim 1, furthercomprising a light absorbing layer disposed over the opticallytransparent spacer and abutting the second mirror.
 8. The opticalfingerprint sensor module of claim 1, wherein the light source comprisesa laser diode or a vertical-cavity surface-emitting laser (VCSEL). 9.The optical fingerprint sensor module of claim 1, further comprising aprotection layer disposed over the photodiode array.
 10. The opticalfingerprint sensor module of claim 9, wherein the protection layercomprises a bandpass filter.
 11. The optical fingerprint sensor moduleof claim 9, wherein the optically transparent spacer comprises an airgap.
 12. The optical fingerprint sensor module of claim 9, wherein theoptically transparent spacer and the protection layer are configured tohave a first index of refraction and a second index of refraction,respectively, so as to reduce image distortion at the surface of thephotodiode array.
 13. The optical fingerprint sensor module of claim 1,wherein the member comprises a pinhole substrate or an aperture plate,and the pinhole is formed on the pinhole substrate or the apertureplate.
 14. An optical fingerprint sensor module to be disposed under anopaque border of a display screen for detecting a fingerprint pattern ofa finger placed adjacent a fingerprint sensing area of the displayscreen, the optical fingerprint sensor module comprising: a photodiodearray having a surface; an optically transparent spacer disposed overthe surface of the photodiode array; a first mirror configured toreflect signal light scattered or reflected off of the finger andtransmitted through the display screen; a lens configured to receive andrefract the signal light reflected by the first mirror, the lens havingan optical axis that forms an angle with respect to a normal of thesurface of the photodiode array that is between 45 degrees and 135degrees; a member defining a pinhole disposed behind the lens, thepinhole configured to transmit the signal light refracted by the lens; asecond mirror disposed behind the pinhole and above the opticallytransparent spacer, the second mirror configured to reflect the signallight transmitted through the pinhole toward the surface of thephotodiode array; and electronic circuitries electrically coupled to thephotodiode array, wherein the photodiode array is configured to convertthe signal light incident thereon into electrical signals, and theelectronic circuitries are configured to process the electrical signalsto produce an image of a fingerprint pattern of the finger.
 15. Theoptical fingerprint sensor module of claim 14, wherein the pinhole ispositioned off of the optical axis of the lens.
 16. The opticalfingerprint sensor module of claim 14, wherein the angle between theoptical axis of the lens and the normal of the surface of the photodiodearray is between 80 degrees and 95 degrees.
 17. The optical fingerprintsensor module of claim 16, wherein a distance between the optical axisof the lens and the surface of the photodiode array is greater than adistance between the optical axis of the lens and the second mirror. 18.The optical fingerprint sensor module of claim 14, further comprising alight extinction region disposed adjacent the pinhole and abutting thesecond mirror, the light extinction region configured to attenuate aportion of the signal light transmitted through the pinhole and incidenton the light extinction region.
 19. The optical fingerprint sensormodule of claim 14, further comprising a light absorbing layer disposedover the optically transparent spacer and abutting the second mirror.20. The optical fingerprint sensor module of claim 14, wherein themember comprises a pinhole substrate or an aperture plate, and thepinhole is formed on the pinhole substrate or the aperture plate.
 21. Anelectronic device comprising: a display screen including a fingerprintsensing area and an opaque border; a light source configured to provideillumination light directed toward a finger placed adjacent thefingerprint sensing area, a portion of the illumination light beingscattered or reflected off of the finger, thereby generating signallight to be transmitted through the display screen; and an opticalfingerprint sensor module positioned below the display screen under theopaque border, the optical fingerprint sensor module comprising: aphotodiode array having a surface; an optically transparent spacerdisposed over the surface of the photodiode array; a first mirrorconfigured to reflect the signal light; a lens configured to receive andrefract the signal light reflected by the first mirror, the lens havingan optical axis that forms an angle with respect to a normal of thesurface of the photodiode array that is between 45 degrees and 135degrees; a member defining a pinhole disposed behind the lens, thepinhole configured to transmit the signal light refracted by the lens; asecond mirror disposed behind the pinhole and above the opticallytransparent spacer, the second mirror configured to reflect the signallight transmitted through the pinhole toward the surface of thephotodiode array; and electronic circuitries electrically coupled to thephotodiode array, wherein the photodiode array is configured to convertthe signal light incident thereon into electrical signals, and theelectronic circuitries are configured to process the electrical signalsto produce an image of a fingerprint pattern of the finger.
 22. Theelectronic device of claim 21, wherein the pinhole is positioned off ofthe optical axis of the lens.
 23. The electronic device of claim 21,wherein the angle between the optical axis of the lens and the normal ofthe surface of the photodiode array is between 80 degrees and 95degrees.
 24. The electronic device of claim 23, wherein a distancebetween the optical axis of the lens and the surface of the photodiodearray is greater than a distance between the optical axis of the lensand the second mirror.
 25. The electronic device of claim 21, whereinthe optical fingerprint sensor module further comprises a lightextinction region disposed adjacent the pinhole and abutting the secondmirror, the light extinction region configured to attenuate a portion ofthe signal light transmitted through the pinhole and incident on thelight extinction region.
 26. The electronic device of claim 21, whereinthe optical fingerprint sensor module further comprises a lightabsorbing layer disposed over the optically transparent spacer andabutting the second mirror.
 27. The electronic device of claim 21,wherein the member comprises a pinhole substrate or an aperture plate,and the pinhole is formed on the pinhole substrate or the apertureplate.